perm filename CLCOMP[COM,LSP] blob
sn#871210 filedate 1989-03-15 generic text, type T, neo UTF8
∂01-Mar-89 1030 CL-Compiler-mailer Re: Issue WITH-COMPILATION-UNIT
Received: from ti.com by SAIL.Stanford.EDU with TCP; 1 Mar 89 10:30:34 PST
Received: by ti.com id AA02957; Wed, 1 Mar 89 12:18:04 CST
Received: from Kelvin by tilde id AA06738; Wed, 1 Mar 89 12:13:45 CST
Message-Id: <2813767988-8661469@Kelvin>
Sender: GRAY@Kelvin.csc.ti.com
Date: Wed, 1 Mar 89 12:13:08 CST
From: David N Gray <Gray@DSG.csc.ti.com>
To: "Kim A. Barrett" <IIM%ECLA@ECLC.USC.EDU>
Cc: kmp@SCRC-STONY-BROOK.ARPA, cl-compiler@SAIL.STANFORD.EDU
Subject: Re: Issue WITH-COMPILATION-UNIT
In-Reply-To: Msg of Sat 18 Feb 89 19:17:10-PST from Kim A. Barrett <IIM%ECLA@ECLC.USC.EDU>
> What I'd like is to make the syntax of the macro be something like
>
> WITH-COMPILATION-UNIT var &body body
>
> and add something like the following to it description
>
> The body is processed with var bound to a compile-time environment. The
> extent of the environment is the dynamic extent of the form. This extent also
> applies to any environments made from it (directly or indirectly) by
> AUGMENT-ENVIRONMENT (see Issue SEMANTIC-ENVIRONMENT-ACCESS).
This is an interesting idea, but I am afraid it is mixing two things
that don't necessarily go together. In the Lisp Machine's MAKE-SYSTEM,
the environment for compile-time definitions is for each individual
file, while the warnings context includes all of the files.
> COMPILE-FILE needs a :environment argument.
Yes, if we have AUGMENT-ENVIRONMENT, then users might want to be able to
compile within an environment of macro definitions that they
constructed. It should be easy to permit this, although it might not be
so easy to specify the limitations on how to use it properly.
∂02-Mar-89 1037 CL-Compiler-mailer issue LOAD-TIME-EVAL, version 10
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 2 Mar 89 10:37:42 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA04847; Thu, 2 Mar 89 11:23:49 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA03273; Thu, 2 Mar 89 11:23:35 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903021823.AA03273@defun.utah.edu>
Date: Thu, 2 Mar 89 11:23:33 MST
Subject: issue LOAD-TIME-EVAL, version 10
To: cl-compiler@sail.stanford.edu
Cc: moon@stony-brook.scrc.symbolics.com
Here is the writeup on this issue that incorporates the language
suggested by Moon. I've made that into an alternate proposal, and the
writeup includes both the new proposal and the one that was voted on
at the last meeting.
Forum: Compiler
Issue: LOAD-TIME-EVAL
References: #, (p. 356), (EVAL-WHEN (LOAD) ...) (p. 69-70)
issue SHARP-COMMA-CONFUSION
Category: ADDITION
Edit history: 06-Jun-87, Version 1 by James Kempf
17-Jul-87, Version 2 by James Kempf
12-Nov-87, Version 3 by Pitman (alternate direction)
01-Feb-88, Version 4 by Moon
(from version 2 w/ edits suggested by Masinter)
06-Jun-88, Version 5 by Pitman
(fairly major overhaul, merging versions 3 and 4)
21-Sep-88, Version 6 by Moon (stripped down)
17-Oct-88, Version 7 by Loosemore (change direction again)
30-Dec-88, Version 8 by Loosemore (tweaks)
23-Jan-89, Version 9 by Loosemore (amendments)
02-Mar-89, Version 10 by Loosemore (new proposal)
Problem description:
Common Lisp provides reader syntax (#,) which allows the programmer
to designate that a particular expression within a program is to be
evaluated early (at load time) but to later be treated as a constant.
Unfortunately, no access to this capability is available to programs
which construct other programs without going through the reader.
Some computations can be deferred until load time by use of EVAL-WHEN,
but since EVAL-WHEN must occur only at toplevel, and since the nesting
behavior of EVAL-WHEN is quite unintuitive, EVAL-WHEN is not a general
solution to the problem of load-time computation of program constants.
Proposal R**2-NEW-SPECIAL-FORM was approved at the January 1989
meeting. After the meeting, some additional suggestions were made that
have been incorporated into proposal R**3-NEW-SPECIAL-FORM.
Proposal (LOAD-TIME-EVAL:R**2-NEW-SPECIAL-FORM):
Add a new special form, LOAD-TIME-VALUE, which has the following
contract:
LOAD-TIME-VALUE form &optional read-only-p [Special Form]
LOAD-TIME-VALUE provides a mechanism for delaying evaluation of <form>
until the expression is in the "runtime" environment.
If a LOAD-TIME-VALUE expression is seen by COMPILE-FILE, the compiler
performs normal semantic processing such as macro expansion but
arranges for the evaluation of <form> to occur at load time in a null
lexical environment, with the result of this evaluation then being
treated as an immediate quantity at run time. It is guaranteed that
the evaluation of <form> will take place only once when the file is
loaded, but the order of evaluation with respect to the "evaluation"
of top-level forms in the file is unspecified.
If a LOAD-TIME-VALUE expression appears within a function compiled
with COMPILE, the <form> is evaluated at compile time in a null lexical
environment. The result of this compile-time evaluation is treated as
an immediate quantity in the compiled code.
In interpreted code, <form> is evaluated (by EVAL) in a null
lexical environment, and one value is returned. Implementations which
implicitly compile (or partially compile) expressions passed to
EVAL may evaluate the <form> only once, at the time this
compilation is performed. This is intentionally similar to the
freedom which implementations are given for the time of expanding
macros in interpreted code.
Note that, in interpreted code, there is no guarantee as to when
evaluation of <form> will take place, or the number of times the
evaluation will be performed. Since successive evaluations of the
same LOAD-TIME-VALUE expression may or may not result in an evaluation
which returns a "fresh" object, destructive side-effects to the
resulting object may or may not persist from one evaluation to the
next. It is safest to explicitly initialize the object returned by
LOAD-TIME-VALUE, if it is later modified destructively.
Implementations must guarantee that each reference to a
LOAD-TIME-VALUE expression results in at least one evaluation of its
nested <form>. For example,
(DEFMACRO CONS-SELF (X)
`(CONS ,X ,X))
(CONS-SELF (LOAD-TIME-VALUE (COMPUTE-IT)))
must perform two calls to COMPUTE-IT; although there is only one
unique LOAD-TIME-VALUE expression, there are two distinct references
to it.
In the case of a LOAD-TIME-VALUE form appearing in a quoted expression
passed to EVAL, each call to EVAL must result in a new evaluation of
<form>. For example,
(DEFVAR X 0)
(DEFUN FOO () (EVAL '(LOAD-TIME-VALUE (INCF X))))
is guaranteed to increment X each time FOO is called, while
(DEFUN FOO () (LOAD-TIME-VALUE (INCF X)))
may cause X to be evaluated only once.
The READ-ONLY-P argument designates whether the result can be considered
read-only constant. If NIL (the default), the result must be considered
ordinary, modifiable data. If T, the result is a read-only quantity
which may, as appropriate, be copied into read-only space and/or shared
with other programs. (Because this is a special form, this argument is
-not- evaluated and only the literal symbols T and NIL are permitted.)
Rationale:
LOAD-TIME-VALUE is a special form rather than a function or macro
because it requires special handling by the compiler.
Requiring the compiler to perform semantic processing such as macro
expansion on the nested <form>, rather than delaying all such processing
until load time, has the advantages that fewer macro libraries may need
to be available at load time, and that loading may be faster and result
in less consing due to macroexpansion. If users really want to delay
macroexpansion to load time, this can be done with an explicit call to
EVAL, e.g.
(LOAD-TIME-VALUE (EVAL '(MY-MACRO)))
Allowing the same LOAD-TIME-VALUE to cause its nested <form> to be
evaluated more than once makes simplifies its implementation in
interpreters which do not perform a preprocessing code walk. It also
makes the rules for the time of its processing analogous to those
for macro expansion.
This proposal explicitly does -not- tie LOAD-TIME-VALUE to the #,
read macro. Doing so would be an incompatible change to the definition
of #, (which is reliably useful only -inside- quoted structure,
while LOAD-TIME-VALUE must appear -outside- quoted structure in a
for-evaluation position).
The requirement that LOAD-TIME-VALUE expressions be evaluated once per
reference (rather than once per unique expression) prevents problems
that could result by performing destructive side-effects on a value
that is unexpectedly referenced in more than one place.
Proposal (LOAD-TIME-EVAL:R**3-NEW-SPECIAL-FORM):
Add a new special form, LOAD-TIME-VALUE, which has the following
contract:
LOAD-TIME-VALUE form &optional read-only-p [Special Form]
LOAD-TIME-VALUE provides a mechanism for delaying evaluation of <form>
until the expression is in the "runtime" environment.
If a LOAD-TIME-VALUE expression is seen by COMPILE-FILE, the compiler
performs normal semantic processing such as macro expansion but
arranges for the evaluation of <form> to occur at load time in a null
lexical environment, with the result of this evaluation then being
treated as an immediate quantity at run time. It is guaranteed that
the evaluation of <form> will take place only once when the file is
loaded, but the order of evaluation with respect to the "evaluation"
of top-level forms in the file is unspecified.
If a LOAD-TIME-VALUE expression appears within a function compiled
with COMPILE, the <form> is evaluated at compile time in a null lexical
environment. The result of this compile-time evaluation is treated as
an immediate quantity in the compiled code.
In interpreted code, <form> is evaluated (by EVAL) in a null
lexical environment, and one value is returned. Implementations which
implicitly compile (or partially compile) expressions passed to
EVAL may evaluate the <form> only once, at the time this
compilation is performed. This is intentionally similar to the
freedom which implementations are given for the time of expanding
macros in interpreted code.
If the same (compared with EQ) list (LOAD-TIME-VALUE <form>) is
evaluated or compiled more than once, it is unspecified whether <form>
is evaluated only once or is evaluated more than once. This can
happen both when an expression being evaluated or compiled shares
substructure, and when the same expression is passed to EVAL or to
COMPILE multiple times. Since a LOAD-TIME-VALUE expression may be
referenced in more than one place and may be evaluated multiple times
by the interpreter, it is unspecified whether each execution returns
a "fresh" object or returns the same object as some other execution.
Users must use caution when destructively modifying the resulting
object.
If two lists (LOAD-TIME-VALUE <form>) are EQUAL but not EQ, their
values always come from distinct evaluations of <form>. Coalescing
of these forms is not permitted.
The READ-ONLY-P argument designates whether the result can be considered
read-only constant. If NIL (the default), the result must be considered
ordinary, modifiable data. If T, the result is a read-only quantity
which may, as appropriate, be copied into read-only space and/or shared
with other programs. (Because this is a special form, this argument is
-not- evaluated and only the literal symbols T and NIL are permitted.)
Rationale:
LOAD-TIME-VALUE is a special form rather than a function or macro
because it requires special handling by the compiler.
Requiring the compiler to perform semantic processing such as macro
expansion on the nested <form>, rather than delaying all such processing
until load time, has the advantages that fewer macro libraries may need
to be available at load time, and that loading may be faster and result
in less consing due to macroexpansion. If users really want to delay
macroexpansion to load time, this can be done with an explicit call to
EVAL, e.g.
(LOAD-TIME-VALUE (EVAL '(MY-MACRO)))
Allowing the same LOAD-TIME-VALUE to cause its nested <form> to be
evaluated more than once makes simplifies its implementation in
interpreters which do not perform a preprocessing code walk. It also
makes the rules for the time of its processing analogous to those
for macro expansion.
This proposal explicitly does -not- tie LOAD-TIME-VALUE to the #,
read macro. Doing so would be an incompatible change to the definition
of #, (which is reliably useful only -inside- quoted structure,
while LOAD-TIME-VALUE must appear -outside- quoted structure in a
for-evaluation position).
Allowing multiple references to the same LOAD-TIME-VALUE expression
to result in only one interpretation allows it to be specified more
cleanly. It also allows interpreters that do not perform a prepass
to cache LOAD-TIME-VALUE expressions.
Current Practice:
This is an addition to the language and has not yet been implemented.
Cost to Implementors:
In compiled code, (LOAD-TIME-VALUE <form>) is similar to
'#,<form>. Most implementations can probably make use of the same
mechanism they use to handle #, to handle LOAD-TIME-VALUE. Note that
#, does not currently provide a mechanism for dealing with
non-read-only-ness.
Implementing LOAD-TIME-VALUE in the interpreter should be fairly
straightforward, since one simply needs to evaluate the <form> in the
null lexical environment. Implementations that use a preprocessing
code walk in the interpreter to perform macro expansion could process
LOAD-TIME-VALUE forms at that time.
Some code-walkers would have to be taught about this new
special form. Such changes would likely be trivial.
Cost to Users:
Some code-walkers would have to be taught about this new
special form. Such changes would likely be trivial.
Benefits:
Users are given a mechanism that to force evaluation to be delayed
until load time that does not rely on a feature of the reader.
Discussion:
Earlier versions (up to version 7) of this proposal stated that
all semantic processing of the LOAD-TIME-VALUE form should be postponed
until load time.
The semantics of LOAD-TIME-VALUE would be simplified considerably if
the READ-ONLY-P argument were removed and destructive operations on
the result of evaluating <form> prohibited. However, some people feel
that the ability to destructively modify the value is an essential
feature to include.
"Collapsing" of multiple references to the same LOAD-TIME-VALUE
expression could be allowed for read-only situations, but it seems
like it would be more confusing to make it legal in some situations
and not in others.
A number of other alternatives have been considered on this issue,
including:
- A proposal for a new special form that would force evaluation of
the <form> to happen only once. This was rejected because of
implementation difficulties.
- A proposal to add a function making the "magic cookie" used by #,
available to user code. The current proposal does not prevent such
a function from being added, but this approach appeared to have
less support than making the hook available as a new special form.
- A proposal to remove #, entirely (issue SHARP-COMMA-CONFUSION).
- A suggestion to change the behavior of (EVAL-WHEN (LOAD) ...).
Kent Pitman says:
Although I'm willing to take multiple evaluation in the interpreter
as a compromise position, I would like it mentioned in the discussion
that this was only an expedient to getting this issue accepted at all,
and that I'm not really happy about it. I have said that I think a
number of our lingering problems (with EVAL-WHEN, COMPILER-LET, and
this -- for example) are due to the presence of interpreters which do
not do a semantic-prepass at a known time. If I had my way, we would
require a semantic pre-pass and we would then be able to forbid
multiple evaluations even in the interpreter.
Moon and Gray support proposal R**3-NEW-SPECIAL-FORM.
Pitman also expressed willingness to go along with
R**3-NEW-SPECIAL-FORM, but was somewhat concerned that coalescing
LOAD-TIME-VALUE results based on EQ-ness of the LOAD-TIME-VALUE form
could conceivably lead to trouble down the line. However, since he
could provide no actual examples to back up that worry, and since the
majority opinion was that some implementations would find a
restriction against such coalescing an undue burden, the decision was
made to just `note the concern' and proceed on. Sandra Loosemore and
JonL White concur with this position.
-------
∂03-Mar-89 0841 CL-Compiler-mailer Re: Issue LOAD-TIME-EVAL (shouldn't this be called issue
Received: from RELAY.CS.NET by SAIL.Stanford.EDU with TCP; 3 Mar 89 08:40:53 PST
Received: from relay2.cs.net by RELAY.CS.NET id aa00173; 3 Mar 89 8:46 EST
Received: from draper.com by RELAY.CS.NET id aa03622; 3 Mar 89 8:41 EST
Date: Fri, 3 Mar 89 07:47 EST
From: "Steve Bacher (Batchman)" <SEB1525@draper.com>
Subject: Re: Issue LOAD-TIME-EVAL (shouldn't this be called issue
LOAD-TIME-VALUE?)
To: cl-compiler@SAIL.STANFORD.EDU
X-VMS-To: CL-COMPILER,SEB1525
> Some computations can be deferred until load time by use of EVAL-WHEN,
> but since EVAL-WHEN must occur only at toplevel, and since the nesting
> behavior of EVAL-WHEN is quite unintuitive, EVAL-WHEN is not a general
> solution to the problem of load-time computation of program constants.
"must occur only at toplevel"? That isn't true, is it? (In any case,
whether the nesting behavior of EVAL-WHEN is unintuitive may be resolved
by the outcome of the EVAL-WHEN-NON-TOP-LEVEL issue - then again, it may
be unintuitive no matter what is decided. :-) )
> If a LOAD-TIME-VALUE expression is seen by COMPILE-FILE, the compiler
> performs normal semantic processing such as macro expansion but
"Normal semantic processing" includes transformation into machine code or
whatever the compiler normally does with code, right? This should be made
clearer, lest it be assumed that only macroexpansion is permitted.
∂05-Mar-89 1604 CL-Compiler-mailer deadlines
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 5 Mar 89 16:04:38 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA01383; Sun, 5 Mar 89 17:02:30 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA05604; Sun, 5 Mar 89 10:54:30 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903051754.AA05604@defun.utah.edu>
Date: Sun, 5 Mar 89 10:54:29 MST
Subject: deadlines
To: cl-compiler@sail.stanford.edu
If you have comments or suggestions on -any- of the pending
cl-compiler issues, please get them to me by this coming Friday (the
10th). These things need to be sent out to X3J13 early next week to
meet the 2-week deadline, and this will give me a couple of days to
make a final pass over the writeups.
I expect that on a few of the issues we may need to do one more
iteration between next week and the meeting, particularly if we get a
lot of suggestions back from the wider community.
-Sandra
-------
∂05-Mar-89 1653 CL-Compiler-mailer subcommittee meeting
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 5 Mar 89 16:53:07 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA02196; Sun, 5 Mar 89 17:50:46 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA05731; Sun, 5 Mar 89 17:50:44 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903060050.AA05731@defun.utah.edu>
Date: Sun, 5 Mar 89 17:50:42 MST
Subject: subcommittee meeting
To: cl-compiler@sail.stanford.edu
Cc: cperdue@sun.com
> Date: Sun, 5 Mar 89 14:13:08 PST
> From: cperdue@Sun.COM (Cris Perdue)
>
> Do you plan to hold a compiler subcommittee meeting on Monday
> March 27? If so, what time?
I wasn't planning to have a subcommittee meeting this time, because it
seemed like many people indicated they would not be able to make a
Monday meeting anyway. I will, however, be available Monday evening,
if anybody has something they want to discuss informally. Would people
be interested in getting together for dinner, say around 6ish?
We are tentatively on the agenda for Wednesday afternoon, BTW.
-Sandra
-------
∂05-Mar-89 1742 CL-Compiler-mailer issue CONSTANT-COMPILABLE-TYPES, version 7
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 5 Mar 89 17:42:10 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA02978; Sun, 5 Mar 89 18:40:07 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA05764; Sun, 5 Mar 89 18:40:04 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903060140.AA05764@defun.utah.edu>
Date: Sun, 5 Mar 89 18:40:02 MST
Subject: issue CONSTANT-COMPILABLE-TYPES, version 7
To: cl-compiler@sail.stanford.edu
Thanks to Cris Perdue, here is a brand new version of this issue.
There have been a few major changes in content since the version that
was distributed prior to the January meeting:
- All mention of cross-compilation has gone away.
- The behavior of readtables, methods, and generic functions is now
unspecified. Functions are still in, but see the note I added to the
discussion section.
- Error terminology has changed. The proposal used to say "is an error"
but now requires implementations either to do something useful or
signal an error. If anybody has a problem with this, speak up.
Forum: Compiler
Issue: CONSTANT-COMPILABLE-TYPES
References: CLtL pp. 56, 77-80, 324
Issue CONSTANT-MODIFICATION
Issue CONSTANT-CIRCULAR-COMPILATION
Issue CONSTANT-ARRAY-ATTRIBUTES
Issue QUOTE-SEMANTICS
Issue LOAD-OBJECTS
Category: CLARIFICATION, ADDITION
Edit history: 11/07/88, V1 by Cris Perdue
11/14/88, V2 by Cris Perdue
11/22/88, V3 by Cris Perdue
12/20/88, V4 by Cris Perdue
01/06/89, V5 by Sandra Loosemore (minor editorial
clarifications, expand discussion)
03/05/89, V6 by Cris Perdue (more response to comments,
especially from Moon and and from Loosemore)
03/05/89, V7 by Loosemore (more editorial tweaks)
Status: Ready for release
Problem description:
CLtL does not specify what objects can be in compiled constants and it
does not say what relationship there is to be between the
constant passed to the compiler and the one that is established by
compiling and then loading its file. Relevant remarks in CLtL
concerning file compilation and the definition of QUOTE suggest that
the compiler handles constants in ways that are not actually possible.
Introduction to the proposal:
The proposal CONSTANT-COMPILABLE-TYPES:SPECIFY attempts to spell out
what types can appear in compiled constants, and what it means when
they appear. Unless stated otherwise, in this proposal where the term
"constant" is used, it means a quoted or self-evaluating constant, not
a named (defconstant) constant.
The key is a definition of a form of equivalence between Lisp objects,
"similarity as constants". Code passed through the file compiler and
then loaded must behave as though quoted constants in it are "similar"
to quoted constants in the corresponding interpreted "source" code.
Because it is legitimate to compile in one address space and load into
a different one, it is necessary for the constraints to be defined
across address spaces. This proposal only concerns quoted constants
to be processed by COMPILE-FILE. Some other issues related to file
compilation are CONSTANT-COLLAPSING, CONSTANT-CIRCULAR-COMPILATION,
and QUOTE-SEMANTICS.
Some implementations "lose information" about some constants during
compilation. Typically all constant arrays become simple arrays
during the process of compiling and loading. We try to balance the
desire for more functionality against the effort required from
implementors.
Comments within the text of the proposal are enclosed in double angle
brackets, <<like this>>.
Proposal: CONSTANT-COMPILABLE-TYPES:SPECIFY
An object may be used as a quoted constant processed by COMPILE-FILE
if the compiler can guarantee that the resulting constant established
by loading the compiled file is "similar as a constant" to the
original. Treatment of uninterned symbols must be consistent across
the entire file as described below. There also is a constraint on
arrays which is not symmetrical; compilation can make arrays
"simpler", but not "less simple". (See below for the definition.)
We refer below to "quoted constants" or just "constants". In this
section these terms refer to objects appearing in expressions of the
form (QUOTE <object>), to objects used as self-evaluating forms, and
to objects appearing in code at locations described as "not
evaluated".
Some types such as streams are not supported in constants. Put
another way, an object containing one of these is not considered
similar as a constant to any other object. Some implementations may
support them and define how they are treated. For any object that
appears in a constant, but is not supported by the language as part of
a constant, the behavior of the compiler is unspecified; either the
the compiler and/or loader will handle that constant (in an
implementation-dependent manner) or the compiler will detect the
situation and signal an error.
Of the types supported in constants, some are treated as aggregate
objects. For these types, being similar as constants is defined
recursively. We say that an object of these types has certain
attributes, and to be similar as a constant to another object, the
values of the corresponding attributes of the two objects must also be
similar as constants.
This kind of definition has problems with any circular or "infinitely
recursive" object such as a list that is an element of itself. We
use the idea of depth-limited comparison, and say that two
objects are similar as constants if they are similar at all finite
levels. This idea is implicit in the definitions below, and applies
in all the places where attributes of two objects are required to be
similar as constants.
Here we define the notion of two objects being "similar as constants",
organizing the definition by type, and note additional constraints
that the compiler and loader working together must meet:
Number
If either of the two objects is a number, both must be of the same
type and must represent the same mathematical value.
Character
If either of the two objects is a character, both must be character
objects that represent the same character. <<Note that this
definition has to depend on the results of the Character Set
proposals.>>
Random-state
Let us say that two random-states are functionally equivalent if
applying RANDOM to them repeatedly always produces the same
pseudo-random numbers in the same order.
Two random-states are similar as constants exactly if copies of them
made via MAKE-RANDOM-STATE are functionally equivalent.
Note that a constant random-state object cannot be used as the "state"
argument to the function RANDOM (because RANDOM side-effects this
data structure).
Symbol
A symbol can only be similar to a symbol. References to interned
symbols are "by name". <<See issue COMPILE-FILE-SYMBOL-HANDLING for
details.>>
If a symbol is not interned, i.e. its home package is NIL, it is
treated in a rather special way. To be similar as a constant to
another symbol, both symbols must be uninterned and have the same
name.
Constants that contain uninterned symbols have to satisfy an extra
constraint. Consider the set of places in a constant that refer to
the same (EQ) uninterned symbol. In any similar constant, the
corresponding places must also all be EQ -- no more places and no
fewer. Moreover, COMPILE-FILE must arrange for the EQness of all
constant uninterned symbols that appear in the file to be preserved,
even if they are referenced in separate constants.
Because hash keys can be aggregate objects and because we treat hash
tables as unordered sets of <key, value> pairs, similarity of hash
tables is more complex. See under "Hash Tables", below, for the
definition.
Package
A package can only be similar as a constant to a package. References
to packages are permitted in any constant. References to packages are
"by name": two packages are similar as constants when their names are
similar as constants. Within a Lisp "address space", packages with
the same name are EQ.
At load time, the package becomes the same as returned by
FIND-PACKAGE, given the package name. An error is signalled if no
package of that name exists at load time.
AGGREGATE TYPES
---------------
For each of the types listed below, if either object is of the given
type, the two objects are similar as constants exactly if the other is
of that type and the values of all of the specified attributes are
similar as constants.
The attributes listed here can be called the "Basic Attributes" of
objects of each of these types.
Cons CAR, CDR.
Array For 1-dimensional arrays:
LENGTH, ARRAY-ELEMENT-TYPE, and ELT for all legal indices.
For arrays of other dimensions:
ARRAY-DIMENSIONS, ARRAY-ELEMENT-TYPE, AREF for all legal
indices.
An array of type SIMPLE-ARRAY can only be similar as a
constant to an array of type SIMPLE-ARRAY. However, we
allow the file-compiler a bit of latitude here. Where
constants in source code are displaced, have fill
pointers, or are adjustable, constants in the code
resulting from compilation and loading are permitted to
lack any or all of these qualities.
Hash Table Keys and value pairs. The table's test is unchanged
also. If the file compiler is given a constant containing a
a hash table that has keys that are similar as
constants, the consequences are undefined.
Consider a hash table as an unordered set of key and
value pairs. Two hash tables are similar as constants
exactly if there is a one-to-one correspondence between
the key and value pairs of each and a one-to-one
correspondence between the uninterned symbols of each
such that the two keys of each corresponding pair are
similar as constants and the two values are also similar
as constants. The correspondence of uninterned symbols
must be consistent with the correspondence defined for
the entire set of constants in the file.
Pathname Each pathname component.
OTHER TYPES
-----------
Stream, Compiled-Function, Readtable, Generic-function, Method
Objects of these types are not supported in compiled
constants.
Function Only function constants that are not compiled-functions
and do not close over any (lexical) variables are
supported in compiled constants.
Two such functions are similar as constants if their
SOURCE-LAMBDA-EXPRESSIONs are similar as constants.
Structure, Standard-object
<<There is a cl-cleanup issue, LOAD-OBJECTS, pending
which proposes a mechanism for dealing with objects.>>
For structure instances with no method defined at compile
time for MAKE-LOAD-FORM, the slot values and the name of
structure type (a symbol reference) are recorded by the
compiler and reconstructed by the loader.
Examples:
If source code contains a constant that could PRINT as (#1=#:FOO
#2=#:FOO #1# #2#), the constant resulting from compiling and loading
that code would have to be PRINTable as (#1=#:FOO #2=#:FOO #1# #2#).
If we make a hash table H, set three variables A, B, and C to
different uninterned symbols named FOO, and enter keys and values as
follows:
(setf (gethash a h) b)
(setf (gethash b h) a)
(setf (gethash c h) c)
If H appears in a compiled constant, after compiling and loading it,
(let ((value (list)))
(maphash #'(lambda (x y) (push (list x y) value)) h)
value)
could print as
((#1=#:FOO #2=#:FOO) (#2# #1#) (#3=#:FOO #3#))
but not as
((#1=#:FOO #2=#:FOO) (#2# #3=#:FOO) (#3# #1#))
Rationale:
For the benefit of users, this proposal tries to define a fairly large
set of types that all Common Lisp implementations are to handle. It
also attempts to leave room for implementations to differ. Some
implementations have made opposing choices because the language
doesn't specify one over the other. Some implementations already
handle constants that this proposal defines as not legal in Common
Lisp programs, and that is useful to users of those systems.
Different implementors have different amounts of resources to apply to
their system, and implementations differ in their whole approach in
some cases.
This proposal appears to reflect user demand and appears not to exceed
the capabilities of most implementations of the language.
The proposal ensures that all references to the same uninterned symbol
within a file will all map to references to just one uninterned symbol
after compiling and loading. This is needed to support PCL.
Current practice:
>From Gail Zacharias (Nov 14): "Coral pretty much implements this
proposal (I think we currently coalesce hash table keys, but that's
just a bug that will be fixed). We also fasdump packages (using the
package name) and compiled functions (but not foreign functions). For
symbols, we dump the name, and if (roughly speaking) the symbol would
get printed with a package prefix, we also dump the package name and
load the symbol into that package (otherwise it gets loaded into the
current load-time package)."
>From David Gray (Nov 9): "The Explorer can compile constant functions,
read tables, and hash tables; an error is signalled for a stream. A
package object used to break the compiler but in release 5 it has been
fixed to generate instructions to call FIND-PACKAGE on the package
name at load time." (Nov 15): [The Explorer does not guarantee
retention of displaced-to and displaced-index-offset attributes.]
"The Explorer also does not currently support dumping closures (either
compiled or evaluated), although non-closure compiled functions can be
dumped."
>From David Moon (Jan 24): "Symbolics Genera current practice: aside
from some current bugs we have with circular structures of certain
types and with preserving the identity of CONSes under EQ, this is
more or less consistent with our current practice, if you made the
changes implied by my earlier comments. We preserve the :displaced-to
and :fill-pointer array attributes. I doubt that we do what the
proposal says for hash-tables, readtables, and random-states. We
support dumping compiled and interpreted functions, but not closures,
which in effect means we don't support dumping functions."
>From Sandra Loosemore (Mar 3): "UCL currently can handle only
constants that are of type number, character, symbol, cons,
simple-vector, or string (which it turns into simple-string). It
signals an error if an attempt is made to compile any other kind of
object as a constant."
Adoption cost:
Not known. Probably moderate or low -- for most implementations. The
cost would be to implementors rather than users since this part of the
language is currently underspecified. The author believes the cost
will be reasonable for KCL, an implementation where there is some
concern about this issue.
This proposal is close to compatible with the Franz, Lucid, Coral,
Texas Instruments, and Symbolics implementations. It is probably
compatible or nearly compatible with other "Lisp Machine"
implementations.
Benefits:
Users would be able to use aggregate objects in constants with
confidence about the behavior of their code.
Conversion cost:
Where this proposal *requires* different behavior than an existing
implementation, there is a conversion cost for users of that
implementation. It appears that this cost will be small, less than
the cost of leaving things unspecified.
Esthetics:
Since there is no adequate definition at present, a fuller definition
would be more esthetic.
Discussion:
This proposal does leave some user-visible attributes of objects
unspecified across the compile-and-load process, except that they must
be consistent with the attributes that must be retained. This
situation is a compromise between the desire for full specification on
the one hand, and on the other hand the desire to leave freedom for
different implementations to remain different and to support some
optimizations such as compacting hash tables and "simplifying" arrays.
Proposals will be entertained for tighter specification of datatypes
such as arrays.
The full extension of the concept of coalescing of constants is to say
that they can be coalesced exactly when they are similar as constants.
Comparing functions semantically is intertwined with the specification
of what conforming programs and implementations are allowed to do.
This proposal does not attempt to do that since compiled functions are
not supported by this proposal in compiled constants.
The definition of similarity for random-states supports the
possibility of random states that are immutable because of being in
compiled constants.
Readtables need not be supported by an implementation. If a readtable
contains only symbols to represent functions, here is Cris Perdue's
suggested spec for similarity of readtables:
Character syntax type for each character in the table;
function for each readmacro character, mappings for
dispatch macros; whether terminating or nonterminating
for each readmacro.
Interest has been expressed by a number of people including users, in
support for user-definable "dumping" of CLOS objects and structure
instances. The cleanup issue LOAD-OBJECTS deals with this.
This subsumes the issue CONSTANT-ARRAY-ATTRIBUTES.
Sandra Loosemore says:
I plan to submit an amendment to this proposal which would remove the
requirement that the compiler be able to dump non-compiled, non-closed
functions. The reason for removing this requirement is that there is
no way to portably construct an object which is guaranteed to be a
non-compiled, non-closed function. Note that implementations are
permitted to make all functions COMPILED-FUNCTIONs.
JonL White notes:
The analogy between FIND-PACKAGE and FIND-CLASS suggests that class
objects are in the same "database" category as packages. Shouldn't
they be referenced "by name" in compiled file?
Moon responds:
In Flavors we generate metaobjects at compile time, but we never put
them (to speak loosely) into the compiled-code file; instead macros
like DEFFLAVOR and DEFMETHOD expand into Lisp code that obtains new
metaobjects at load time, based on the class name and generic function
name. I don't see how any other way could work, actually, since two
distinct compiled files that refer to a class by the same name must
end up referring to the same metaobject after loading. In Flavors we
don't have anonymous classes nor anonymous generic functions, so we
don't have to solve those issues.
[Issue LOAD-OBJECTS proposes] that the way to load an instance of a
standard-class from a compiled file is for a method of the instance
to return a form which is then evaluated at load time. The semantics
of loading an instance of a standard-class from a compiled file can be
entirely understood in terms of MAKE-INSTANCE or whatever other
function is called by the form returned by MAKE-LOAD-FORM; no new
concepts need be introduced. If the programmer of a given class wants
to use the class redefinition protocol, that class's MAKE-LOAD-FORM
method can output something that uses that protocol, and if he
doesn't, it can output something that doesn't.
-------
∂05-Mar-89 1819 CL-Compiler-mailer subcommittee meeting
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 5 Mar 89 18:19:38 PST
Received: from BOBOLINK.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 551042; Sun 5-Mar-89 21:16:49 EST
Date: Sun, 5 Mar 89 21:16 EST
From: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>
Subject: subcommittee meeting
To: sandra%defun@CS.Utah.EDU
cc: CL-Compiler@SAIL.Stanford.EDU, cperdue@Sun.COM
In-Reply-To: <8903060050.AA05731@defun.utah.edu>
Message-ID: <890305211639.5.KMP@BOBOLINK.SCRC.Symbolics.COM>
CL-Editorial is tentatively planning to meet Monday evening. If both
met simultaneously, I would probably attend the editorial meeting. My
presence is not an essential part of the meeting, of course, but I
figured I'd mention the conflict -- especially since others might have
it, too.
If you're not reporting until Wednesday, why not plan to meet after
hours Tuesday?
∂05-Mar-89 1837 CL-Compiler-mailer Re: subcommittee meeting
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 5 Mar 89 18:37:37 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA03552; Sun, 5 Mar 89 19:35:30 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA05943; Sun, 5 Mar 89 19:35:26 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903060235.AA05943@defun.utah.edu>
Date: Sun, 5 Mar 89 19:35:23 MST
Subject: Re: subcommittee meeting
To: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>
Cc: CL-Compiler@SAIL.Stanford.EDU, cperdue@Sun.COM
In-Reply-To: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>, Sun, 5 Mar 89 21:16 EST
Yes, a Tuesday evening meeting is also a possibility. Let's try to be
flexible about it until we see what people's travel plans are like and
what (and how many) issues are still causing us trouble.
-Sandra
-------
∂05-Mar-89 1917 CL-Compiler-mailer cl-compiler issue status, as of 3/5
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 5 Mar 89 19:17:25 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA04133; Sun, 5 Mar 89 20:15:24 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA06000; Sun, 5 Mar 89 20:15:21 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903060315.AA06000@defun.utah.edu>
Date: Sun, 5 Mar 89 20:15:18 MST
Subject: cl-compiler issue status, as of 3/5
To: cl-compiler@sail.stanford.edu
Here's a quick summary of active cl-compiler issues.
These issues seem to be in pretty good shape right now. I know some
of them will need minor changes to the writeup before they are
released, mostly relating to updating current practice and tightening
up error language. Some of them haven't gotten much comment since the
last version was distributed.
COMPILE-FILE-SYMBOL-HANDLING
(Except for possible flames from JonL?)
COMPILED-FUNCTION-REQUIREMENTS
COMPILER-DIAGNOSTICS
COMPILER-VERBOSITY
CONSTANT-CIRCULAR-COMPILATION
CONSTANT-COLLAPSING
CONSTANT-COMPILABLE-TYPES
LOAD-TIME-EVAL
MACRO-CACHING
MACRO-ENVIRONMENT-EXTENT
(Except for conflicting reports about what CLOS assumes.)
QUOTE-SEMANTICS
The writeups for the following issues still need major revisions. These
are the ones I personally think are important and am going to try to finish
up in time to be voted on.
CLOS-MACRO-COMPILATION
(New issue on compile-time behavior of CLOS defining macros,
waiting for input from CLOS committee.)
COMPILE-ENVIRONMENT-CONSISTENCY
(The new version of the writeup is complete except for the CLOS
section.)
COMPILER-LET-CONFUSION
(Probably present both ELIMINATE and REPAIR proposals but add Kim's
SYMBOL-MACROLET technique to the presentation.)
DEFINING-MACROS-NON-TOP-LEVEL
(Needs to be updated to match EVAL-WHEN-NON-TOP-LEVEL.)
EVAL-WHEN-NON-TOP-LEVEL
(We need to try to reach closure on this, or agree to present more than
one proposal.)
These are issues where the writeups still need major revisions but
somebody else is going to have to work on them. (I certainly don't
want to discourage anybody from doing so.)
COMPILE-FILE-ENVIRONMENT
DEFCONSTANT-NOT-WIRED
DEFINE-OPTIMIZER
PROCLAIM-ETC-IN-COMPILE-FILE
SYNTACTIC-ENVIRONMENT-ACCESS (aka MACRO-ENVIRONMENT-CREATOR)
WITH-COMPILATION-UNIT
-Sandra
-------
∂06-Mar-89 1622 CL-Compiler-mailer Re: COMPILE-FILE-SYMBOL-HANDLING:HOME-PACKAGE
Received: from Sun.COM by SAIL.Stanford.EDU with TCP; 6 Mar 89 16:22:47 PST
Received: from snail.Sun.COM by Sun.COM (4.1/SMI-4.0)
id AA25042; Mon, 6 Mar 89 11:28:36 PST
Received: from clam.sun.com by snail.Sun.COM (4.1/SMI-4.0)
id AA06794; Mon, 6 Mar 89 11:24:22 PST
Received: by clam.sun.com (4.0/SMI-4.0)
id AA04038; Mon, 6 Mar 89 11:27:43 PST
Date: Mon, 6 Mar 89 11:27:43 PST
From: cperdue@Sun.COM (Cris Perdue)
Message-Id: <8903061927.AA04038@clam.sun.com>
To: sandra%clam@Sun.COM
Subject: Re: COMPILE-FILE-SYMBOL-HANDLING:HOME-PACKAGE
Cc: cl-compiler@sail.stanford.edu
This proposal came out differently than I expected:
"When a compiled file is loaded, the interned symbols it
references are found by calling INTERN at load time with two arguments,
the name of the symbol and the package with the same name as
the compile-time symbol's home package. ... "
This doesn't ensure that the home package remains the same across
compilation and loading, which I consider the key consideration.
How about this statement instead?
"When a file is compiled, the symbol name is recorded together
with the home package name and an indication of whether the
symbol is external in its home package. At load time the
symbol is effectively looked up with:
(find-symbol string (find-package pkg-name))
If the symbol is noted as external, it must be found at load
time as :external. If it is noted as internal, it must either
be present in the package or not found at all. If it is not found
at all, it is created as if by:
(intern string (find-package pkg-name))
If the package system is not in a suitable state, an error is
signalled."
Here is some code that purports to do the right thing:
(defun resolve-symbol (name pkg-name external?)
(multiple-value-bind (sym where)
(find-symbol string (find-package pkg-name))
(if external?
(if (eq where :external) sym <signal error>)
(case where
((:internal :external) sym)
(nil (intern string (find-package pkg-name)))
(otherwise <signal error>)))))
---------------------------
This is what I consider "the right thing". I'd like to see it
given as one of the alternative proposals if it isn't already what
was intended by proposal HOME-PACKAGE.
-Cris
∂06-Mar-89 1711 CL-Compiler-mailer Re: COMPILE-FILE-SYMBOL-HANDLING:HOME-PACKAGE
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 6 Mar 89 17:11:38 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA08531; Mon, 6 Mar 89 18:09:33 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA06924; Mon, 6 Mar 89 18:09:30 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903070109.AA06924@defun.utah.edu>
Date: Mon, 6 Mar 89 18:09:29 MST
Subject: Re: COMPILE-FILE-SYMBOL-HANDLING:HOME-PACKAGE
To: cperdue@Sun.COM (Cris Perdue)
Cc: cl-compiler@sail.stanford.edu
In-Reply-To: cperdue@Sun.COM (Cris Perdue), Mon, 6 Mar 89 11:27:43 PST
What you describe is not what proposal HOME-PACKAGE was intended to
say. It is, however, a behavior that I would consider reasonable. It
has the benefit of offering somewhat tighter error checking, but the
disadvantage that it is even less forgiving of minor changes to the
package structure than proposal HOME-PACKAGE. For example, suppose
you compile your file in package P1 that uses the P2 package, and you
reference a symbol FOO whose home package is P1. If, between compile
and load time, another symbol called FOO is exported from the P2
package, you lose.
If there is a strong belief that this behavior is correct, I don't
have any objection to adding it as another proposal and/or modifying
proposal REQUIRE-CONSISTENCY to allow this as a legitimate
implementation technique, other than that it's a lot of work. :-)
-Sandra
-------
∂06-Mar-89 1711 CL-Compiler-mailer issue LOAD-TIME-EVAL, version 10
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 6 Mar 89 17:10:17 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 551813; Mon 6-Mar-89 20:07:29 EST
Date: Mon, 6 Mar 89 20:07 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: issue LOAD-TIME-EVAL, version 10
To: Sandra J Loosemore <sandra%defun@cs.utah.edu>
cc: cl-compiler@sail.stanford.edu
In-Reply-To: <8903021823.AA03273@defun.utah.edu>
Message-ID: <19890307010708.9.MOON@EUPHRATES.SCRC.Symbolics.COM>
I've read this and have no further comments, except that the
differences between the two proposals would be easier to
understand if they were stated explicitly. It's hard to compare
two multi-page pieces of text without using a computer.
∂06-Mar-89 1812 CL-Compiler-mailer draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 6 Mar 89 18:11:53 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 551857; Mon 6-Mar-89 21:09:20 EST
Date: Mon, 6 Mar 89 21:08 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
To: Sandra J Loosemore <sandra%defun@cs.utah.edu>
cc: cl-compiler@sail.stanford.edu, Moon@STONY-BROOK.SCRC.Symbolics.COM
In-Reply-To: <8902242327.AA11369@defun.utah.edu>
Message-ID: <19890307020856.2.MOON@EUPHRATES.SCRC.Symbolics.COM>
Date: Fri, 24 Feb 89 16:27:57 MST
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Here is a draft of an alternate EVAL-WHEN proposal.
I am open to suggestions on this writeup. If everybody else hates
this and thinks that the the general direction here is totally wrong,
I will shut up and go along with the GENERALIZE-EVAL proposal. But I
personally would like to see something like this included in the
writeup we send out to X3J13 to vote on.
I know I'm not "everybody else," but I think the general direction here
is wrong. Here's your rationale:
Rationale:
This behavior specified by this proposal is simple and easy to
understand, and extends the behavior of EVAL-WHEN usefully to
non-top-level situations. It is largely compatible with the behavior
of EVAL-WHEN specified in CLtL, except for the situation shown in
example #6 above. Some people find the nesting behavior of EVAL-WHEN
specified in this proposal to be an improvement over that specified
in CLtL.
This gives a useful meaning to EVAL-WHEN that supports useful and
predictable behavior if defining macros are used in a non-top-level
situation.
It's true that this extends to non-top-level situations, but I don't
believe that is either simple or easy to understand, as compared with
what CLtL says (or the version 5 proposal by Pitman and Moon that
extends CLtL to non-top-level situations). I suppose that is a matter
of taste, but here's a specific example: In CLtL, if you want to make
a top-level form do what it always does and also happen at compile
time, you wrap (eval-when (eval load compile) ...) around it. That
works. In this proposal, eval-when always makes its body non-top-level,
which means that it is impossible to make a top-level form do what
it always does, plus something else, by wrapping an eval-when around it.
To me that makes it complicated and hard to understand. I also think
the new shielded/unshielded concept is obfuscation.
Aside from that, I am very leary about the idea of changing a part of
the language that works in a way that some people don't like, to work in
a different way that other people don't like, when the semantic issues
are sufficiently complicated and cloudy that nobody can predict the
repercussions of the change. This just seems very dangerous. I would
be the first to admit that CLtL's eval-when feature is not the most
beautiful and well-thought-out language feature to come down the pike.
It also has a remarkably badly chosen set of keywords, which I think
contributes to confusion in the discussion. However, your proposed
alternative doesn't seem to be much better, and suffers from the
disadvantage of not having a user community with five years' experience.
I would much rather be offered the choice of keeping EVAL-WHEN
compatible or switching to something different and obviously correct.
But I think the choice here is between keeping EVAL-WHEN compatible and
switching to something that's just different.
However, here's my real problem with all this. I'm trying to figure out
what the standard should say about the expansion of the CLOS
defining-form macros. I know that they should be defined in terms of
EVAL-WHEN. The continuing instability of EVAL-WHEN, and the large
amount of mail to plow through, is really interfering with my ability to
figure anything out. That may well say more about me than about
EVAL-WHEN, but it's still a fact. I wish I could force myself to ignore
everything you're doing and just use the CLtL definition of EVAL-WHEN
for now.
∂06-Mar-89 1840 Common-Lisp-Object-System-mailer remote environments
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 6 Mar 89 18:39:50 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 551871; Mon 6-Mar-89 21:37:29 EST
Date: Mon, 6 Mar 89 21:37 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: remote environments
To: David N Gray <Gray@DSG.csc.ti.com>
cc: Common-Lisp-Object-System@SAIL.Stanford.edu, CL-Compiler@SAIL.Stanford.edu
In-Reply-To: <2813272681-7847725@Kelvin>
Message-ID: <19890307023706.3.MOON@EUPHRATES.SCRC.Symbolics.COM>
Date: Thu, 23 Feb 89 18:38:01 CST
From: David N Gray <Gray@DSG.csc.ti.com>
I think it might help to focus the discussion about remote environments
and meta object programming if we had a clearer picture of what the
goals are. The basic question is what kinds of things can be defined and
then used during compilation of the same file that defines them, and
what restrictions might apply.
I think you're right. Let me offer my opinions on the set of issues
you have articulated.
DEFCLASS
* Can the class be used as a superclass of a later DEFCLASS? [clearly yes]
* Can it be used as a specializer in a DEFMETHOD? [clearly yes]
Agreed.
* Can a MAKE-INSTANCE be done by a macro expander, DEFCONSTANT, or "#."?
I have two answers to this, both based on current practice but
contradictory. One is that I have some users who really need to be able
to instantiate such classes in macro expanders. They have an embedded
language which has an object-oriented representation for programs;
therefore macro expansions include instances of classes defined earlier
in the same file. The second is that Flavors does not support
instantiation of compile-time classes in the Symbolics implementation,
consequently the users I mentioned are currently operating with a kludge.
I think it would be much nicer if we could make compile-time classes
instantiable. However, I agree that it would not ruin the language to
omit that feature if we can't figure out how to do it.
- If so, do initforms have access to macros and constants defined
earlier in the file?
Initforms certainly have access to those things since they are included
in the initforms' environment. I think 88-002R implies this.
* Can the class be used as the :METACLASS option of a later DEFCLASS?
- Can that second class be instantiated?
* Can it be used as the :GENERIC-FUNCTION-CLASS option of a
DEFGENERIC, GENERIC-FUNCTION, GENERIC-FLET, or GENERIC-LABELS?
* Can it be used as the :METHOD-CLASS option of a DEFGENERIC etc.?
- Can DEFMETHODs then be done for that generic function?
The answers to these three should be the same and should depend just on
whether a remote (aka compile-time) class can be instantiated. If yes,
then the subsidiary questions are clearly also yes.
DEFGENERIC
* Referenced by later DEFMETHOD? [clearly yes]
* Is the function defined such that it can be called at compile time?
Clearly not, since a DEFUN is not. Here I think we should defer to the
definition of how COMPILE-FILE deals with DEFUN and not try to propose
something "better" that is just for CLOS. In fact I do have something
better in mind, in which COMPILE-FILE would be less different from
normal Lisp evaluation. But I don't think it would be appropriate to
propose something so radical for Common Lisp at its current life stage.
DEFMETHOD
* Can it be invoked at compile-time?
* In particular, will methods added to standard generic functions be
invoked by the system at compile time?
No, and no, for the same reason.
When you compile-file a DEFMETHOD, a method metaobject is created but it
is not added to the generic-function metaobject in the local environment.
Instead it is added to a different generic-function metaobject created
in the remote environment. That's my model of what has to happen. Note
that this should be completely consistent with the way that compile-file
of a DEFCLASS, with a direct superclass whose name is defined in the
local environment and not in the remote environment, does not add the
new class metaobject to the direct subclasses of the local superclass,
but rather to a different object. (I realize we haven't agreed on what
this paragraph says, or even seen a coherent proposal, yet. I'm just
telling you my model.)
DEFINE-METHOD-COMBINATION
* Used in a later DEFGENERIC?
- Callable at compile-time?
I believe this should be yes to both, although if I'm not mistaken
Flavors does not allow it. I think that's a bad design choice in
Flavors.
Are there other interactions that need to be considered?
I can make a few other points. Assuming remote classes can be
instantiated, remote methods specialized to a remote class of course
cannot be executed. However, nothing stops us from making a local
method specialized to a remote class. There happens not to be a
defmethod syntax for doing that [although in fact one could imagine
such an extension], but it should be easy to do with the interface
at the next level down. That's a benefit from the clear separation
between names and objects for which CLOS is striving.
In the past there has been some controversy about whether the remote
environment can inherit from the local environment. I think this is
crystal clear: since some user-defined classes have STANDARD-OBJECT
as a direct superclass, and STANDARD-OBJECT is not defined in the same
file, the remote environment is clearly inheriting from the local
environment. Different implementations might want to address the
details of this differently, but I think it's clear that there has to
be provision for it in the metaobject model. It makes things more
complicated, but that's unavoidable.
I think that the standard could take a simple, minimal, approach that
would still satisfy the most common usages. Suppose we said:
DEFCLASS
If it appears at top-level, then the class name is defined for use as
a type specifier or method specializer. It can also be used as a
superclass of a later DEFCLASS since they don't have to be defined
before being referenced anyway. The class object can be obtained by
calling FIND-CLASS with an environment argument, but it can only be
used in ways that do not require the class to be finalized. For
example, one could ask for its CLASS-DIRECT-SUPERCLASSES, but not its
CLASS-PRECEDENCE-LIST. Other uses, which could involve the need to
instantiate the class, could not be portably done in the same file
without wrapping an (EVAL-WHEN (EVAL COMPILE LOAD) ...) around the
DEFCLASS. Implementations would be free to support compile-time
instantiation as an extension. One way to look at this would be to
say that it is implementation-dependent whether FINALIZE-INHERITANCE
works or signals an error when given a class defined in the
compile-time environment.
[And no compile-time generic-function or method objects at all]
This is an interesting idea, but I think it's too restrictive. Here's a
plausible and many-times proposed application for metaobjects which
would not be possible if we adopted this idea. Suppose you made an
optimizing compiler that is allowed to assume that no class
redefinitions, no method redefinitions, and no newly-defined subclasses
will be created at run time. The compiler is to take advantage of this
constraint on the program to generate more efficient code by doing type
propagation and constant-folding out many method lookups and slot
lookups. One should expect many CLOS programs compiled this way to have
the same efficiency as C++ without suffering the same restrictions
during development. Now, the natural way to organize the datastructures
in this compiler is as metaobjects. CLOS (chapter 3 at least) already
defines how to access the information the compiler needs. The
constraint against run-time redefinition means the compiler can assume
certain functions of metaobjects return the same result at compile time
as they must at run time. For this to work all the metaobjects must
exist and finalization must possible. It doesn't appear that
instantiation is required, assuming the program being compiled doesn't
define any metaclasses.
I hope to keep thinking in this direction.
∂06-Mar-89 2146 CL-Compiler-mailer Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 6 Mar 89 21:46:33 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA15544; Mon, 6 Mar 89 22:44:22 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA07259; Mon, 6 Mar 89 22:44:01 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903070544.AA07259@defun.utah.edu>
Date: Mon, 6 Mar 89 22:43:59 MST
Subject: Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
To: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Cc: Sandra J Loosemore <sandra%defun@cs.utah.edu>,
cl-compiler@sail.stanford.edu
In-Reply-To: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>, Mon, 6 Mar 89 21:08 EST
Your concerns have been noted. In response, I want to say that the
reason why I've been pursuing this is that I just don't feel
completely comfortable with the GENERALIZE-EVAL proposal and I haven't
entirely given up hope of coming up with something better. That you
need such a large piece of pseudocode, two state variables, plus a
table to describe the nesting behavior to specify how EVAL-WHEN works
bothers me because it's an indication of just how complicated the
semantics are. The semantics make as much sense as any of the other
alternatives we've considered, but I still wish we could come up with
a less complicated model of how EVAL-WHEN behaves.
On the other hand, I realize that my alternate proposal doesn't
represent a vast improvement and that we are running out of time to
come up with radically different alternatives. If those of us who
still have reservations about the GENERALIZE-EVAL proposal haven't
reached consensus (or at least major progress in that direction) on an
alternative by the end of the week, there won't be one on the ballot.
Please, go ahead and formulate your model of how the CLOS macros work
in terms of the GENERALIZE-EVAL proposal. Once we have something
specific in hand there, it will be easier for all of us to understand
what the interactions between the two issues might be.
-Sandra
-------
∂06-Mar-89 2243 CL-Compiler-mailer Re: Issue: EVAL-WHEN-NON-TOP-LEVEL (Version 5)
Received: from ti.com by SAIL.Stanford.EDU with TCP; 6 Mar 89 22:42:33 PST
Received: by ti.com id AA03025; Mon, 6 Mar 89 12:19:38 CST
Received: from Kelvin by tilde id AA05183; Mon, 6 Mar 89 12:06:37 CST
Message-Id: <2814199584-13822129@Kelvin>
Sender: GRAY@Kelvin.csc.ti.com
Date: Mon, 6 Mar 89 12:06:24 CST
From: David N Gray <Gray@DSG.csc.ti.com>
To: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>
Cc: CL-Compiler@SAIL.Stanford.EDU, Moon@STONY-BROOK.SCRC.Symbolics.COM
Subject: Re: Issue: EVAL-WHEN-NON-TOP-LEVEL (Version 5)
In-Reply-To: Msg of Fri, 24 Feb 89 14:08 EST from Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>
> I wasn't intending my :EXECUTE to be different from your :EXECUTE. I'm
> not sure if I just used a poor choice of words to describe it or if I've
> missed some subtlety of your proposal.
>
> LOAD and COMPILE have meaning only when in a non-top-level position.
> EVAL has meaning only in a top-level position.
OK, I see now that I did miss noticing the sentence in your proposal that
says:
The use of EVAL controls whether processing occurs for
non-top-level forms.
But this just makes the proposal even more incompatible that I thought
before.
> For any given context, if LOAD is `being considered,' then COMPILE is
> `being considered' -- and vice versa.
>
> For any given context, if LOAD and COMPILE are `being considered' then
> EVAL is not `being considered' -- and vice versa.
...
I found this discussion to be incomprehensible until I did a text
substitution to replace EVAL, COMPILE, and LOAD with your alternate names
:EXECUTE, :COMPILE-TOPLEVEL, and :LOAD-TOPLEVEL. Then it began to make
sense. It just isn't going to work to define new meanings for old words;
if this approach is going to be taken, then new names are essential.
Also, to help make this clearer, instead of :EXECUTE, it should be
something like :EXECUTE-NON-TOP-LEVEL.
Now that I understand the proposal, I still need to think about whether I
agree with it.
∂06-Mar-89 2354 CL-Compiler-mailer Issue: SAFE-CODE (Version 1)
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 6 Mar 89 23:54:11 PST
Received: from BOBOLINK.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 551956; Tue 7-Mar-89 02:51:39 EST
Date: Tue, 7 Mar 89 02:51 EST
From: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Issue: SAFE-CODE (Version 1)
To: CL-Compiler@SAIL.Stanford.EDU
Message-ID: <890307025131.8.KMP@BOBOLINK.SCRC.Symbolics.COM>
Issue: SAFE-CODE
Forum: Compiler
References: OPTIMIZE declaration (p160),
Issue ERROR-TERMINOLOGY
Category: CLARIFICATION/CHANGE
Edit history: 07-Mar-89, Version 1 by Pitman
Status: For Internal Discussion
Problem Description:
The new error terminology refers to ``safe code'' in the definition
of the term and CLtL refers to
individual meanings of OPTIMIZE qualities, but there is no standardized
way of relating the two concepts.
Proposal (SAFE-CODE:SAFETY-3):
Define that, formally, the term ``safe code'' is code refers to any
code in which the OPTIMIZE quality for SAFETY has a value of 3.
Implementors might wish to consider treating other situations as safe
as well, but in making that decision both the relative values of other
OPTIMIZE qualities and the idiosyncratic properties of the particular
implementation should also be taken into account.
Examples:
1. The body of the following is safe...
a. (LOCALLY (DECLARE (OPTIMIZE (SAFETY 3))) . body)
b. (LOCALLY (DECLARE (OPTIMIZE SAFETY )) . body)
2. The body in each of the following is unsafe. They might
or might not be treated as safe, possibly depending
on the values of other qualities and specifics of the
implementation.
a. (LOCALLY (DECLARE (OPTIMIZE (SAFETY 0))) . body)
b. (LOCALLY (DECLARE (OPTIMIZE (SAFETY 1))) . body)
c. (LOCALLY (DECLARE (OPTIMIZE (SAFETY 2))) . body)
Rationale:
Programmers will probably intuitively expect that the term
``highest safety'' refers to giving the SAFETY quality its
highest safety.
Current Practice:
Implementors ...
Symbolics Genera does error checking always, and ignores OPTIMIZE
declarations.
Symbolics Cloe heeds OPTIMIZE declarations, but effectively makes
`judgment calls' in every case because there is no clear guidance
on how to interpret them.
Programmers ...
Many programmers write (DECLARE (SPEED 0) (SAFETY 3)) even when all
they really want to control is SAFETY because they are afraid that
unless they explicitly sacrifice speed, the compiler will ignore
their plea for error checking.
Cost to Implementors:
Some implementations might require a lot of nitpicky little changes.
Cost to Users:
Technically none. No portable code can really rely on much of any
reliable effect out of any of the OPTIMIZE qualities. However, some
users may rely on implementation-specific features of implementations,
and if those implementations are forced to change, non-portable user
code might break in some ways.
Cost of Non-Adoption:
The meaning of ``safe code'' will not be clearly defined.
Benefits:
Programmers will be able to say what they mean. They can stop
superstitiously putting (SPEED 0) next to (SAFETY 3) just to
assure they get safe code.
Aesthetics:
Improved. This will make the English align well with the code.
Discussion:
It is very important that we reach consensus in some form on this issue.
Pitman supports SAFE-CODE:SAFETY-3.
∂07-Mar-89 0727 CL-Compiler-mailer Re: Issue: SAFE-CODE (Version 1)
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 7 Mar 89 07:27:32 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA25432; Tue, 7 Mar 89 08:25:28 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA07523; Tue, 7 Mar 89 08:25:22 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903071525.AA07523@defun.utah.edu>
Date: Tue, 7 Mar 89 08:25:20 MST
Subject: Re: Issue: SAFE-CODE (Version 1)
To: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>
Cc: CL-Compiler@SAIL.Stanford.EDU
In-Reply-To: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>, Tue, 7 Mar 89 02:51 EST
Actually, I was one of the people who thought this was "intuitively
obvious", but an explicit statement sounds like a good idea. Thanks
for writing this up.
-Sandra
-------
∂07-Mar-89 0808 CL-Compiler-mailer Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
Received: from multimax.encore.com by SAIL.Stanford.EDU with TCP; 7 Mar 89 08:06:20 PST
Received: from mist.encore.COM by multimax.encore.com with SMTP (5.61/25-eef)
id AA17032; Tue, 7 Mar 89 11:04:40 -0500
Received: from localhost by mist. (4.0/SMI-4.0)
id AA16212; Tue, 7 Mar 89 11:02:48 EST
Message-Id: <8903071602.AA16212@mist.>
To: "David A. Moon" <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Cc: cl-compiler@sail.stanford.edu
Subject: Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
In-Reply-To: Your message of Mon, 06 Mar 89 21:08:00 -0500.
<19890307020856.2.MOON@EUPHRATES.SCRC.Symbolics.COM>
Date: Tue, 07 Mar 89 11:02:42 EST
From: Dan L. Pierson <pierson@mist.encore.com>
Date: Mon, 6 Mar 89 21:08 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
However, here's my real problem with all this. I'm trying to figure out
what the standard should say about the expansion of the CLOS
defining-form macros. I know that they should be defined in terms of
EVAL-WHEN. The continuing instability of EVAL-WHEN, and the large
amount of mail to plow through, is really interfering with my ability to
figure anything out. That may well say more about me than about
EVAL-WHEN, but it's still a fact. I wish I could force myself to ignore
everything you're doing and just use the CLtL definition of EVAL-WHEN
for now.
You're right, this has dragged on much too long. Since
GENERALIZE-EVAL-NEW-KEYWORDS in Version 5 seems to be the best
proposal so far, I support it. I'm not as opposed to Sandra's new
proposal as you are; but since I also don't see any major advantages
to it, the incompatibility isn't worth it.
I would very much like to see what GENERALIZE-EVAL-NEW-KEYWORDS will
do to DEFINING-MACROS-NON-TOP-LEVEL before voting for the former. I
still want to be able to wrap a LET around a couple of DEFUNs and have
it all work correctly!
∂07-Mar-89 0828 CL-Compiler-mailer issue DEFINING-MACROS-NON-TOP-LEVEL
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 7 Mar 89 08:28:15 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA27258; Tue, 7 Mar 89 09:26:05 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA07624; Tue, 7 Mar 89 09:26:03 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903071626.AA07624@defun.utah.edu>
Date: Tue, 7 Mar 89 09:26:01 MST
Subject: issue DEFINING-MACROS-NON-TOP-LEVEL
To: Dan L. Pierson <pierson@mist.encore.com>
Cc: cl-compiler@sail.stanford.edu
In-Reply-To: Dan L. Pierson <pierson@mist.encore.com>, Tue, 07 Mar 89 11:02:42 EST
> Date: Tue, 07 Mar 89 11:02:42 EST
> From: Dan L. Pierson <pierson@mist.encore.com>
>
> I would very much like to see what GENERALIZE-EVAL-NEW-KEYWORDS will
> do to DEFINING-MACROS-NON-TOP-LEVEL before voting for the former. I
> still want to be able to wrap a LET around a couple of DEFUNs and have
> it all work correctly!
Right. I think we are all agreed that the rules for when defining
macros cause compile-time magic should be exactly the same as those
for when (EVAL-WHEN (COMPILE) ...) causes compile-time magic,
precisely so that you can implement the defining macros using
EVAL-WHEN.
The only major change I'm planning to do the the existing writeup is
to move the definition of "top-level" versus "non-top-level" to issue
EVAL-WHEN-NON-TOP-LEVEL. I think it really makes more sense there,
since that is where the COMPILE-FILE model is presented.
-Sandra
-------
∂07-Mar-89 0912 CL-Compiler-mailer Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 7 Mar 89 09:11:58 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 552137; Tue 7-Mar-89 12:09:05 EST
Date: Tue, 7 Mar 89 12:08 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
To: Dan L. Pierson <pierson@mist.encore.com>
cc: cl-compiler@sail.stanford.edu, Moon@STONY-BROOK.SCRC.Symbolics.COM
In-Reply-To: <8903071602.AA16212@mist.>
Message-ID: <19890307170826.9.MOON@EUPHRATES.SCRC.Symbolics.COM>
Date: Tue, 07 Mar 89 11:02:42 EST
From: Dan L. Pierson <pierson@mist.encore.com>
I would very much like to see what GENERALIZE-EVAL-NEW-KEYWORDS will
do to DEFINING-MACROS-NON-TOP-LEVEL before voting for the former. I
still want to be able to wrap a LET around a couple of DEFUNs and have
it all work correctly!
That's difficult to answer, since the last version of
EVAL-WHEN-NON-TOP-LEVEL archived here is version 7, which the compiler
committee said in Kauai they were going to rewrite. However, here is
the proposal section from version 7, with my comments interleaved.
I will ignore the new-keywords versus keep-the-old-names issue, as
that clearly affects only the appearance of code, not the semantics.
Proposal: DEFINING-MACROS-NON-TOP-LEVEL:ALLOW
(1) Remove the language from p. 66 of CLtL quoted above. Clarify that
while defining macros normally appear at top level, it is meaningful
to place them in non-top-level contexts and that the compiler must
handle them properly in all situations. However, the compile-time side
effects described in issue COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS
only take place when the defining macros appear at top-level.
True under all EVAL-WHEN proposals.
(2) Remove the language on p. 145 of CLtL, which states that macro
functions are always defined in the null lexical environment. Clarify
that all defining macros which create functional objects (including
DEFMACRO, DEFTYPE, DEFINE-SETF-METHOD, and the complex form of
DEFSETF, as well as DEFUN) must ensure that those functions are
defined in the lexical environment in which the defining form is
evaluated.
True under all EVAL-WHEN proposals except for the one (no longer extant
I believe) that said that the body of an EVAL-WHEN is executed in the
null lexical environment rather than the lexical environment surrounding
the EVAL-WHEN.
(3) Define a ``top-level form'' as follows:
- Each form read by COMPILE-FILE from the input file is a top-level
form.
- Forms within the body of a top-level PROGN, EVAL-WHEN, or
COMPILER-LET are also top-level forms.
- The expansion of a top-level macro call is also a top-level form.
True under the EVAL-WHEN proposal that I like. The second bullet is
false under the EVAL-WHEN proposal that Sandra likes, since the body of
a top-level EVAL-WHEN is not a top-level form under that proposal.
Top-level forms would be evaluated by the interpreter in a null
lexical environment, but evaluation in a null lexical environment does
not necessarily imply that the form is top-level.
True under all EVAL-WHEN proposals.
(4) Specify that top-level forms in a file being compiled are
guaranteed to be processed sequentially, including forms within the
body of a top-level PROGN, EVAL-WHEN, or COMPILER-LET. The order in
which non-top-level subforms of a top-level form are processed by the
compiler is explicitly left unspecified.
Not affected by EVAL-WHEN proposals.
So I think the effect on DEFINING-MACROS-NON-TOP-LEVEL is none other
than avoiding breaking it by violating its assumptions about EVAL-WHEN.
Of course if there is a rewritten DEFINING-MACROS-NON-TOP-LEVEL that
I have not seen (I am not on the compiler committee), the story might
be different.
∂07-Mar-89 0924 CL-Compiler-mailer draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
Received: from lucid.com by SAIL.Stanford.EDU with TCP; 7 Mar 89 09:24:22 PST
Received: from blacksox ([192.9.201.39]) by heavens-gate.lucid.com id AA03343g; Tue, 7 Mar 89 09:17:00 PST
Received: by blacksox id AA01657g; Tue, 7 Mar 89 09:21:49 PST
Date: Tue, 7 Mar 89 09:21:49 PST
From: Eric Benson <eb@lucid.com>
Message-Id: <8903071721.AA01657@blacksox>
To: Moon@STONY-BROOK.SCRC.Symbolics.COM
Cc: pierson@mist.encore.com, cl-compiler@sail.stanford.edu,
Moon@STONY-BROOK.SCRC.Symbolics.COM
In-Reply-To: David A. Moon's message of Tue, 7 Mar 89 12:08 EST <19890307170826.9.MOON@EUPHRATES.SCRC.Symbolics.COM>
Subject: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
Date: Tue, 7 Mar 89 12:08 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
(3) Define a ``top-level form'' as follows:
- Each form read by COMPILE-FILE from the input file is a top-level
form.
- Forms within the body of a top-level PROGN, EVAL-WHEN, or
COMPILER-LET are also top-level forms.
Should we include forms within the body of a top-level MACROLET? This
seems reasonable. What about LOCALLY? This also seems reasonable,
modulo the question of whether LOCALLY is a macro or a special form.
Is this a slippery slope?
∂07-Mar-89 0932 CL-Compiler-mailer Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 7 Mar 89 09:32:06 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA29828; Tue, 7 Mar 89 10:29:37 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA07705; Tue, 7 Mar 89 10:29:20 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903071729.AA07705@defun.utah.edu>
Date: Tue, 7 Mar 89 10:29:19 MST
Subject: Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
To: Eric Benson <eb@lucid.com>
Cc: Moon@STONY-BROOK.SCRC.Symbolics.COM, pierson@mist.encore.com,
cl-compiler@sail.stanford.edu, Moon@STONY-BROOK.SCRC.Symbolics.COM
In-Reply-To: Eric Benson <eb@lucid.com>, Tue, 7 Mar 89 09:21:49 PST
> Should we include forms within the body of a top-level MACROLET? This
> seems reasonable. What about LOCALLY? This also seems reasonable,
> modulo the question of whether LOCALLY is a macro or a special form.
> Is this a slippery slope?
I think we reached consensus at the last meeting that both MACROLET and
SYMBOL-MACROLET should pass through toplevelness.
LOCALLY is still a sticky issue. I tend to agree with JonL that since
it can add things like SPECIAL declarations to the environment, its
body should never be considered toplevel.
-Sandra
-------
∂07-Mar-89 0942 CL-Compiler-mailer draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
Received: from lucid.com by SAIL.Stanford.EDU with TCP; 7 Mar 89 09:42:46 PST
Received: from blacksox ([192.9.201.39]) by heavens-gate.lucid.com id AA03379g; Tue, 7 Mar 89 09:35:43 PST
Received: by blacksox id AA01661g; Tue, 7 Mar 89 09:40:29 PST
Date: Tue, 7 Mar 89 09:40:29 PST
From: Eric Benson <eb@lucid.com>
Message-Id: <8903071740.AA01661@blacksox>
To: sandra%defun@cs.utah.edu
Cc: Moon@STONY-BROOK.SCRC.Symbolics.COM, pierson@mist.encore.com,
cl-compiler@sail.stanford.edu, Moon@STONY-BROOK.SCRC.Symbolics.COM
In-Reply-To: Sandra J Loosemore's message of Tue, 7 Mar 89 10:29:19 MST <8903071729.AA07705@defun.utah.edu>
Subject: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Date: Tue, 7 Mar 89 10:29:19 MST
LOCALLY is still a sticky issue. I tend to agree with JonL that since
it can add things like SPECIAL declarations to the environment, its
body should never be considered toplevel.
What's wrong with adding SPECIAL declarations? Why is it different
from adding local macros or symbol macros?
∂07-Mar-89 1101 CL-Compiler-mailer Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
Received: from ALDERAAN.SCRC.Symbolics.COM ([128.81.41.109]) by SAIL.Stanford.EDU with TCP; 7 Mar 89 11:01:05 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by ALDERAAN.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 278791; Tue 7-Mar-89 13:30:24 EST
Date: Tue, 7 Mar 89 13:30 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
To: Sandra J Loosemore <sandra%defun@cs.utah.edu>
cc: Eric Benson <eb@lucid.com>, pierson@mist.encore.com, cl-compiler@sail.stanford.edu
In-Reply-To: <8903071729.AA07705@defun.utah.edu>
Message-ID: <19890307183012.3.MOON@EUPHRATES.SCRC.Symbolics.COM>
Date: Tue, 7 Mar 89 10:29:19 MST
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
> Should we include forms within the body of a top-level MACROLET? This
> seems reasonable. What about LOCALLY? This also seems reasonable,
> modulo the question of whether LOCALLY is a macro or a special form.
> Is this a slippery slope?
I think we reached consensus at the last meeting that both MACROLET and
SYMBOL-MACROLET should pass through toplevelness.
I thought so too (but I might have thought that only because it was
my own position :-).
LOCALLY is still a sticky issue. I tend to agree with JonL that since
it can add things like SPECIAL declarations to the environment, its
body should never be considered toplevel.
Here is the criterion I prefer to use: a form is non-top-level if its
lexical environment cannot be fully constructed at compile time or if it
is not manifestly executed at load time exactly once. In other words,
the form is enclosed in a variable binding, a block, a tagbody, a loop,
or a conditional. The reason I prefer this criterion is that it is
directly related to the reason we have the kludge of top-level forms in
the language in the first place: compile-file's modelling at compile
time of the actions that will occur at load time.
A lexical environment consists of variables, functions, blocks, go tags,
and declarations. Variables, blocks, and go tags cannot be fully
constructed at load time. Declarations can be fully constructed at
compile time, and functions can also be, unless the definition is a
closure that cannot be fully constructed at compile time; but the way
lexical environments nest in Common Lisp prevents that case from arising
except when the lexical environment of the binding already cannot be
fully constructed at compile time. I think function bindings being
fully constructable at compile time depends on the illegality of using
setf to alter a local function binding, otherwise function bindings
would be just like variable bindings.
Since what LOCALLY adds to the lexical environment can be fully
constructed at compile time, its body should be top-level.
We have tacitly chosen to rule out smart compilers that prove a form is
effectively top-level when a naive compiler might think it was
non-top-level. For example, we require all compilers to treat the defun
in
(let ((x 1))
(defun foo (y)
(expt y 2)))
as non-top-level even though a smart compiler might discover that the
binding of x is irrelevant. Similarly we don't allow a smart compiler
to discover that what appears to be a loop is actually executed exactly
once and treat the forms in its body as top-level. Similarly we don't
allow a smart compiler to "optimize"
(eval '(defun ...)) into (defun ...). I think this was a good decision.
We seem to have added an additional criterion, which is that a form is
non-top-level if its value is used. This means any form that is an
argument to a function, as well as forms appearing in certain special
forms such as DEFPARAMETER, are non-top-level. I think that criterion
may be unnecessary but I have no objection to it. I suspect the reason
for this criterion is nothing more profound than that it allows LOAD's
implementation of the top-level forms to not produce values and hence
to not be nestable inside other top-level forms.
∂07-Mar-89 1140 CL-Compiler-mailer Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 7 Mar 89 11:39:59 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA05094; Tue, 7 Mar 89 12:37:14 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA07782; Tue, 7 Mar 89 12:37:08 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903071937.AA07782@defun.utah.edu>
Date: Tue, 7 Mar 89 12:37:07 MST
Subject: Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
To: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Cc: Sandra J Loosemore <sandra%defun@cs.utah.edu>, Eric Benson <eb@lucid.com>,
pierson@mist.encore.com, cl-compiler@sail.stanford.edu
In-Reply-To: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>, Tue, 7 Mar 89 13:30 EST
> Date: Tue, 7 Mar 89 13:30 EST
> From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
>
> Here is the criterion I prefer to use: a form is non-top-level if its
> lexical environment cannot be fully constructed at compile time or if it
> is not manifestly executed at load time exactly once. In other words,
> the form is enclosed in a variable binding, a block, a tagbody, a loop,
> or a conditional.
I would also add "in a FUNCTION special form", although it's unlikely
that anybody would put one directly at top-level. Also, what about
things like CATCH and UNWIND-PROTECT that alter the dynamic
environment?
As other people have noted before, it seems like we are arriving at a
very complicated idea of "top-level" that has very little to do with
people's intuitive notions on the subject. It makes me wonder if
perhaps we shouldn't retreat to a minimalist position where PROGN is
the only special case.
-Sandra
-------
∂07-Mar-89 1418 CL-Compiler-mailer Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 7 Mar 89 14:18:01 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 552433; Tue 7-Mar-89 17:14:33 EST
Date: Tue, 7 Mar 89 17:14 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
To: Sandra J Loosemore <sandra%defun@cs.utah.edu>
cc: Eric Benson <eb@lucid.com>, pierson@mist.encore.com, cl-compiler@sail.stanford.edu
In-Reply-To: <8903071937.AA07782@defun.utah.edu>
Message-ID: <19890307221419.3.MOON@EUPHRATES.SCRC.Symbolics.COM>
Date: Tue, 7 Mar 89 12:37:07 MST
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
> Date: Tue, 7 Mar 89 13:30 EST
> From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
>
> Here is the criterion I prefer to use: a form is non-top-level if its
> lexical environment cannot be fully constructed at compile time or if it
> is not manifestly executed at load time exactly once. In other words,
> the form is enclosed in a variable binding, a block, a tagbody, a loop,
> or a conditional.
I would also add "in a FUNCTION special form", although it's unlikely
that anybody would put one directly at top-level.
I disagree. The immediate "body" of a FUNCTION special form is not a form,
so the concept of "top level form" cannot be relevant to it. The immediate
"body" might be a LAMBDA expression that has forms in its body, but those
inner forms are not manifestly executed exactly once.
Also, what about
things like CATCH and UNWIND-PROTECT that alter the dynamic
environment?
I'd say that these special forms do not manifestly execute their body
exactly once (because they imply throwing out of the middle of the
body), although I admit that there is a fine distinction here.
As other people have noted before, it seems like we are arriving at a
very complicated idea of "top-level" that has very little to do with
people's intuitive notions on the subject.
I vehemently disagree. My message was intended to arrive at a very simple
idea of "top-level" that corresponds exactly to intuitive notions. I was
hoping that this very simple idea would immediately clarify any supposed
issues such as whether the body of a top-level LOCALLY is or is not top-level.
It makes me wonder if
perhaps we shouldn't retreat to a minimalist position where PROGN is
the only special case.
Do we really have to rehearse all the arguments as to why that won't work
once again?
∂07-Mar-89 1423 CL-Compiler-mailer Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 7 Mar 89 14:22:04 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 552440; Tue 7-Mar-89 17:18:50 EST
Date: Tue, 7 Mar 89 17:18 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
To: Sandra J Loosemore <sandra%defun@cs.utah.edu>
cc: Eric Benson <eb@lucid.com>, pierson@mist.encore.com, cl-compiler@sail.stanford.edu
In-Reply-To: <8903071937.AA07782@defun.utah.edu>
Supersedes: <19890307221419.3.MOON@EUPHRATES.SCRC.Symbolics.COM>
Message-ID: <19890307221837.5.MOON@EUPHRATES.SCRC.Symbolics.COM>
Date: Tue, 7 Mar 89 12:37:07 MST
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
> Date: Tue, 7 Mar 89 13:30 EST
> From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
>
> Here is the criterion I prefer to use: a form is non-top-level if its
> lexical environment cannot be fully constructed at compile time or if it
> is not manifestly executed at load time exactly once. In other words,
> the form is enclosed in a variable binding, a block, a tagbody, a loop,
> or a conditional.
I would also add "in a FUNCTION special form", although it's unlikely
that anybody would put one directly at top-level.
I disagree. The immediate "body" of a FUNCTION special form is not a form,
so the concept of "top level form" cannot be relevant to it. The immediate
"body" might be a LAMBDA expression that has forms in its body, but those
inner forms are not manifestly executed exactly once.
Also, what about
things like CATCH and UNWIND-PROTECT that alter the dynamic
environment?
I'd say that these special forms do not manifestly execute their body
exactly once (because they imply throwing out of the middle of the
body), although I admit that there is a fine distinction here.
As other people have noted before, it seems like we are arriving at a
very complicated idea of "top-level" that has very little to do with
people's intuitive notions on the subject.
I vehemently disagree. My message was intended to arrive at a very simple
idea of "top-level" that corresponds exactly to intuitive notions. I was
hoping that this very simple idea would immediately clarify any supposed
issues such as whether the body of a top-level LOCALLY is or is not top-level.
It makes me wonder if
perhaps we shouldn't retreat to a minimalist position where PROGN is
the only special case.
Do we really have to rehearse all the arguments as to why that won't work
once again?
I'm sorry I flamed so much in this message, but this is the type of response
that makes me wonder if the whole X3J13 effort is nothing but an exercise
in futility.
∂07-Mar-89 1458 CL-Compiler-mailer Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 7 Mar 89 14:58:03 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA18436; Tue, 7 Mar 89 15:55:17 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA08178; Tue, 7 Mar 89 15:55:12 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903072255.AA08178@defun.utah.edu>
Date: Tue, 7 Mar 89 15:55:10 MST
Subject: Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
To: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Cc: Sandra J Loosemore <sandra%defun@cs.utah.edu>, Eric Benson <eb@lucid.com>,
pierson@mist.encore.com, cl-compiler@sail.stanford.edu
In-Reply-To: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>, Tue, 7 Mar 89 17:18 EST
> Date: Tue, 7 Mar 89 17:18 EST
> From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
>
> It makes me wonder if
> perhaps we shouldn't retreat to a minimalist position where PROGN is
> the only special case.
>
> Do we really have to rehearse all the arguments as to why that won't work
> once again?
I'm not aware that such arguments have ever been coherently stated. I
just went through all my back mail on this subject and couldn't find
anything on it from you or from anyone else.
Perhaps we need to be reminded that the minimalist position is
actually the status quo -- PROGN is the only special case mentioned by
CLtL. COMPILER-LET got added to our list because it is also treated
as a special case by implementations derived from the MIT Lisp
Machine. MACROLET and SYMBOL-MACROLET got added because some people
thought it would be generally useful.
I question whether the problem with LOCALLY justifies trying to come
up with a model of what toplevelness is that is radically different
than what is specified in CLtL. The example you cited in your
original message was putting a DEFUN form inside of a LOCALLY. That
would still be legitimate Common Lisp regardless of whether or not the
DEFUN is considered to be at top-level, and since we do not require
DEFUN to do any compile-time evaluation, what are you losing?
> I'm sorry I flamed so much in this message, but this is the type of response
> that makes me wonder if the whole X3J13 effort is nothing but an exercise
> in futility.
I know how you feel; I'm also getting very sick and tired of this
issue. We've been going around in circles on it for a year now and we
don't seem any closer to any real understanding or consensus.
-Sandra
-------
∂07-Mar-89 1546 Common-Lisp-Object-System-mailer Re: remote environments
Received: from Xerox.COM by SAIL.Stanford.EDU with TCP; 7 Mar 89 15:46:17 PST
Received: from Semillon.ms by ArpaGateway.ms ; 07 MAR 89 15:39:49 PST
Date: 7 Mar 89 15:39 PST
From: Danny Bobrow <Bobrow.pa@Xerox.COM>
Subject: Re: remote environments
In-reply-to: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>'s message
of Mon, 6 Mar 89 21:37 EST
To: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
cc: David N Gray <Gray@DSG.csc.ti.com>,
Common-Lisp-Object-System@SAIL.Stanford.edu, CL-Compiler@SAIL.Stanford.edu
Message-ID: <890307-153949-8334@Xerox>
I agree with Gray and you. There is further internal evidence in your
message that we will have to allow instances of classes defined in the file
being compiled to be instantiated. You answer for:
DEFINE-METHOD-COMBINATION
* Used in a later DEFGENERIC?
- Callable at compile-time?
Moon: I believe this should be yes to both, although if I'm not
mistaken Flavors does not allow it. I think that's a bad design
choice in Flavors.
I agree. And the metaobject protocol specifies that this
definition can define a new method-combination class.
An instance of this class is used in the DEFGENERIC to effect the
method combination. This implies an ability to instantiate
a newly defined class at compile time to implement this capability.
Another reason to answer the instntiation question YES.
∂07-Mar-89 1744 CL-Compiler-mailer A (new) (old) definition of "top-level"
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 7 Mar 89 17:44:04 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA24207; Tue, 7 Mar 89 18:41:46 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA08237; Tue, 7 Mar 89 18:41:36 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903080141.AA08237@defun.utah.edu>
Date: Tue, 7 Mar 89 18:41:35 MST
Subject: A (new) (old) definition of "top-level"
To: cl-compiler@sail.stanford.edu
Cc: moon@stony-brook.scrc.symbolics.com
I've spent the past few hours digging around through the literature to
try to get a historical perspective on the use of the term
"top-level". Believe it or not, I've actually found a definition of
the term, from the 1st edition of Winston & Horn's LISP book:
A function is said to be defined or used in the top-level environment
if its definition or use is directly demanded by a user, rather
than indirectly by way of the evaluation of one or more other functions.
Historically, it appears that the term "top-level" was first used in
the context of "top-level READ-EVAL-PRINT loop", a context which also
has an implication of direct evaluation. For example, the manuals for
Stanford Lisp 1.6 (PDP-10) and UT Lisp (CDC-6000) talk about EVAL
being the default "top-level function", and use "top-level expression"
to mean an expression read in and evaluated by the top-level loop.
For example, here's what the UT Lisp manual has to say:
A LISP program is usually composed of a simple sequence of LISP
expressions, or forms, to be evaluated. [...] Program execution is
handled by the function EVAL, the usual "top-level function". EVAL is
called to evaluate successive input expressions in the order they
appear.
Likewise, the Franz Lisp manual uses "top-level" as a noun to mean
"top-level loop".
So how does this notion of top-level-ness fit in to Common Lisp? I
think we have to consider the traditional implementation of a Lisp
interpreter as a recursive procedure. What the Winston & Horn
definition implies is that a top-level form is a form that is seen by
the outermost explicit call to EVAL, and that any recursive calls to
evaluate subforms are not top-level.
In Common Lisp, note that the EVAL function itself cannot be defined
recursively because it doesn't take an environment object. (I hope
that's obvious without me needing to present a proof.) Let's suppose
EVAL is implemented by calling a recursive internal function, passing
it the form and a null lexical environment object. Then, any form
that EVAL itself receives as an argument is a top-level form, but any
subforms which are seen only by the internal function are not
top-level.
This definition doesn't have anything to do with what kinds of things
are in the lexical environment or whether or not they can be
determined in advance. It does fit in with something we've discussed
before, namely that top-level-ness implies evaluation in a null
lexical environment.
Now to to extend this to file compilation. LOADing a source file is
defined as reading and evaluating forms from the file. Since there is
direct evaluation going on, each form that is read from the file is
top-level. This notion applies just as well to COMPILE-FILE, which
must arrange for the loader to "evaluate" or "execute" each of the
top-level forms.
Finally, we have implicitly accepted the idea that macroexpansion by
the compiler must preserve semantics. That implies that
macroexpansion should not have any effect on top-level-ness. Thinking
about our simple compiler model that doesn't do much besides walk the
code to expand all macros and print out the results, if it is allowed
to turn the bodies of COMPILER-LETs, MACROLETs, and SYMBOL-MACROLETs
into PROGNs, then that would be an argument for saying that those
three special forms should be treated the same as PROGN as far as
top-level-ness goes. However, that rationale wouldn't cover
constructs like LOCALLY and LABELS (as Moon suggests) unless we modify
our compiler model.
This way of thinking about top-level-ness makes sense to me. How
about the rest of you?
-Sandra
-------
∂07-Mar-89 1825 CL-Compiler-mailer Re: Potential issue: MACRO-SPECIAL-FORMS
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 7 Mar 89 18:24:55 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 552620; Tue 7-Mar-89 21:21:40 EST
Date: Tue, 7 Mar 89 21:21 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Re: Potential issue: MACRO-SPECIAL-FORMS
To: Jeff Dalton <jeff%aiai.edinburgh.ac.uk@NSS.Cs.Ucl.AC.UK>
cc: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>, cl-compiler@sail.stanford.edu,
cl-cleanup@sail.stanford.edu
In-Reply-To: <3256.8902271723@subnode.aiai.ed.ac.uk>
Message-ID: <19890308022127.7.MOON@EUPHRATES.SCRC.Symbolics.COM>
Just a comment: I really believe that CLtL's goal of minimizing the
number of special forms that a code-analyzer has to understand by
relying on macros is misguided and non-achievable. Consider Sandra's
experience, which I believe other people have had in other
implementations, with something that CLtL says is a macro, but the
compiler does a better job of compiling the macro call than of compiling
the nominally equivalent macro expansion. Just consider how well CLtL's
goal has been achieved in practice: not at all.
I think Common Lisp would do better to define everything that "everybody
knows" is a special form to be a special form; this would include all
the control structures, but not SETF. My claim is that this would not
make it any more difficult to do code analysis, because in practice code
analyzers have to know about most of those forms anyway, and because the
extra work to make a typical code analyzer know about a typical special
form, even something as complicated as DO, is usually fairly small,
certainly less work than figuring out what weird macro expansions all
the implementations in the world will use for DO and making sure the
code analyzer works right for each of them. This is even more true when
you use a pattern-driven code analyzer, such as the one I wrote in 1983
and have given to everyone who asked, or the one in Interlisp
MasterScope (which was my model, although I didn't look at the detailed
code).
An alternative position that I'd consider would be to mandate the exact
macro expansion of a number of these forms, not allowing implementations
to deviate. Essentially that would be moving these things into a
portable library. I suppose that might devolve into endles nitpicking
discussions that we don't need.
∂08-Mar-89 1145 CL-Compiler-mailer issue COMPILE-ENVIRONMENT-CONSISTENCY, version 4
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 8 Mar 89 11:44:11 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA19010; Wed, 8 Mar 89 12:42:02 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA08790; Wed, 8 Mar 89 12:41:59 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903081941.AA08790@defun.utah.edu>
Date: Wed, 8 Mar 89 12:41:58 MST
Subject: issue COMPILE-ENVIRONMENT-CONSISTENCY, version 4
To: cl-compiler@sail.stanford.edu
Here's the latest draft. The major change in content here is that I
have removed the item that dealt with CLOS requirements, and added a
requirement to make the handling of type specifiers defined with
DEFCLASS parallel that for DEFTYPE and DEFSTRUCT. I have also tried
to generally clean up the wording of the writeup in accordance with
various suggestions I've received, from Kathy Chapman and others.
Please let me know if there are additional changes you'd like to see.
Forum: Compiler
Issue: COMPILE-ENVIRONMENT-CONSISTENCY
References: CLtL p. 68-69, 143, 321
RAM's "Compiler Cleanup Proposal #3"
Category: CLARIFICATION
Edit History: V1, 2 Sep 1988, Sandra Loosemore (initial draft)
V2, 9 Sep 1988, Sandra Loosemore (incorporate suggestions)
V3, 26 Oct 1988, Sandra Loosemore (add suggestion from Benson)
V4, 08 Mar 1989, Sandra Loosemore (wording changes)
Problem Description:
CLtL does not clearly specify what aspects of the compiletime
environment the compiler (or other preprocessor) may "wire in" to code
being compiled. At what time (compiletime or runtime) are certain
kinds of definitions "captured"? What happens if these definitions
are not consistent at both compile and run times?
Proposal COMPILE-ENVIRONMENT-CONSISTENCY:CLARIFY:
The process of compilation causes certain kinds of information present
in the compiletime environment to be captured and incorporated into
the resulting compiled code. Other kinds of information may not be
captured until the compiled code is actually run.
Specifically:
(1) The following information *must* be present in the compiletime
environment for the program to be compiled correctly. This
information need not also be present in the runtime environment.
(a) In conforming code, macros referenced in the code being compiled
must have been previously defined in the compiletime environment.
The compiler must treat any form that is a list beginning with
a symbol that does not name a macro or special form as a function
call. (This implies that SETF methods must also be available at
compiletime.)
(b) In conforming code, variables that are intended to be bound
specially must be declared SPECIAL in the compiletime environment
before any bindings of that variable are processed by the compiler.
The compiler must treat any binding of an undeclared variable as a
lexical binding.
(2) The compiler *may* incorporate the following kinds of information
into the code it produces, if the information is present in the
compiletime environment and is referenced within the code being
compiled. Except where some other behavior is explicitly stated, when
the compiletime and runtime definitions are different, it is
unspecified which will prevail within the compiled code. In all
cases, the absence of the information at compiletime is not an error,
but its presence may enable the compiler to generate more efficient
code.
(a) The compiler may assume that functions that are defined and
declared INLINE in the compiletime environment will retain the
same definitions at runtime.
(b) The compiler may assume that, within a named function, a
recursive call to a function of the same name refers to the
same function, unless that function has been declared NOTINLINE.
(c) COMPILE-FILE may assume that, in the absence of NOTINLINE
declarations, a call within the file being compiled to a named
function which is defined in that file refers to that function.
(This permits "block compilation" of files.) The behavior of
the program is unspecified if functions are redefined individually
at runtime.
(d) The compiler may assume that the signature (or "contract") of
all built-in Common Lisp functions will not change. In addition,
the compiler may treat all built-in Common Lisp functions as if
they had been proclaimed INLINE.
(e) The compiler may assume that the signature (or "contract") of
functions with FTYPE information available will not change. (See
issue FUNCTION-TYPE-ARGUMENT-TYPE-SEMANTICS.)
(f) The compiler may "wire in" the values of symbolic constants
that have been defined with DEFCONSTANT in the compiletime
environment.
(g) The compiler can assume that type definitions made with DEFTYPE
or DEFSTRUCT in the compiletime environment will retain the same
definition in the runtime environment. It may also assume that
a class defined by DEFCLASS in the compiletime environment will
be defined in the runtime environment in such a way as to have
the same superclasses and metaclass. This implies that
subtype/supertype relationships of type specifiers will not
change between compiletime and runtime. (Note that it is not
an error for an unknown type to appear in a declaration at
compiletime, although it is reasonable for the compiler to
emit a warning in such a case.)
(h) The compiler may assume that if type declarations are present
in the compiletime environment, the corresponding variables and
functions present in the runtime environment will actually be of
those types; otherwise, the runtime behavior of the program is
undefined.
(3) The compiler *must not* make any additional assumptions about
consistency between the compiletime and runtime environments. In
particular:
(a) The compiler may not assume that functions that are defined
in the compiletime environment will retain the either the
same definition or the same signature at runtime, except
in situations (2a) through (2e) above. It is, however,
permissible for the compiler to emit warning messages when
compiling calls to functions that are defined in the compiletime
environment, but where the wrong number or type of arguments
are supplied.
(b) The compiler may not signal an error if it sees a call to a
function that is not defined at compiletime, since that function
may be provided at runtime. Again, it is permissible for the
compiler to emit a warning in these situations.
Rationale:
This proposal generally reflects current practice.
Current Practice:
There don't seem to be any compilers around that do not implement the
provisions of item (1).
For item (2), most compilers (including Lucid) optimize self-recursive
calls by default. Most compilers also opencode data structure
accessors (such as CAR) at some level of optimization, and some code
much more complicated built-in functions inline as well. VaxLisp, for
example, normally compiles MEMBER inline. The Lucid compiler makes
use of type declarations to perform generic-to-specific
transformations on many arithmetic and sequence functions, which is
also a form of inlining. KCL performs block compilation by default,
and Lucid does so under certain conditions.
Cost to implementors:
Unknown, but probably minor.
Cost to users:
Since most implementations appear to be largely in conformance with the
proposal, users should notice little difference.
Benefits:
The presence of a definite specification of what may happen when will
help users structure their programs so they will compile correctly in
all Common Lisp implementations.
Discussion:
Most of the discussion on this issue has been centered on the
terminology describing error situations for item (2). In most cases
where there is an inconsistency between compile-time and run-time, the
results will be unpredictable but harmless. The major exception is
violation of type declarations, which is a "crash-and-burn" error in
many implementations.
There has also been some concern raised that item (3) would prohibit
such things as a cross-compiler that produces standalone programs in
an environment that disallows redefinition of functions. The intent
of this proposal is not to prohibit a compiler from having a magic
switch that imposes additional restrictions on the programs it
compiles, but such a compiler would not be a compiler for Common Lisp.
Several people have expressed reservations about items 2b and 2c, saying
that self-tail-recursion optimization and block compilation should not
be the default behavior of the compiler. Gail Zacharias responds:
This item [2c] has nothing to do with whether anybody does it by
default. The question is whether an end user can take a Common Lisp
program whose internals he's not familiar with, block-compile it, and
be guaranteed that it will continue to function correctly. This item
says that yes, a correct CL program must explicitly indicate what
functions in the source it will redefine at runtime. I don't think
this places such a great burden on the programmer. Without this
provision, only somebody intimately familiar with a program could know
whether it can be safely block-compiled, making block-compilation
useless in the context of portable CL programs.
This thing about "block compilation shouldn't be the default" seems to
come up every time this item is discussed. That's an environment
question and is not addressed by the proposal. The proposal simply
says that block compilation should be legal.
-------
∂08-Mar-89 1401 CL-Compiler-mailer issue COMPILER-LET-CONFUSION, version 6
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 8 Mar 89 14:01:17 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA24990; Wed, 8 Mar 89 14:59:09 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA08869; Wed, 8 Mar 89 14:59:06 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903082159.AA08869@defun.utah.edu>
Date: Wed, 8 Mar 89 14:59:05 MST
Subject: issue COMPILER-LET-CONFUSION, version 6
To: cl-compiler@sail.stanford.edu
This is another one of the issues I had marked as being in need of
major revisions. Kent and I think that this writeup is ready to go.
If anybody else has something to add to the discussion section, let me
know ASAP.
Issue: COMPILER-LET-CONFUSION
Forum: Compiler
References: CLtL p. 112
Category: CHANGE
Edit History: V1, 27 Sep 1988, Sandra Loosemore (initial version)
V2, 04 Oct 1988, Sandra Loosemore (add another example)
V3, 31 Oct 1988, Sandra Loosemore (only proposal ELIMINATE)
V4, 08 Jan 1989, Kent M. Pitman (new alternative)
V5, 09 Jan 1989, Sandra Loosemore (discussion)
V6, 08 Mar 1989, Sandra Loosemore (general updating)
Problem Description:
The description of the COMPILER-LET special form in CLtL is confusing
to many people. There are no examples provided to make it clear how it
is supposed to be used. The only description which is offered is overly
concrete, which have led to confusion about the intent of COMPILER-LET,
and about its implementability.
The intent of COMPILER-LET was to permit information to be communicated
between macros by use of dynamic variables at macroexpansion time.
It was not necessary to the intended uses of COMPILER-LET that such
variables ever be bound at execution time.
Unfortunately, probably because some implementations did not primitively
support COMPILER-LET at the time CLtL was written, an exception was
permitted to make COMPILER-LET `more or less work' in interpreters:
the COMPILER-LET variables were permitted to be bound at execution time.
The problem was further compounded by the fact that CLtL presented this
exception as part of COMPILER-LET's contract rather than as an
implementation note, and by the fact that no examples of actually using
COMPILER-LET correctly are provided.
Subtle bugs can be introduced because of the different handling of the
variable bindings in the interpreter and the compiler. In compiled
code, the bindings are only lexically visible during the expansion of
macros at compile time, while in interpreted code the bindings have
dynamic scope and may also be seen during ordinary evaluation if
evaluation and macroexpansion happen concurrently.
Further compatibility problems can result from the value forms being
evaluated in a null lexical environment in the compiler and the ordinary
lexical environment in the interpreter.
Background and Analysis:
It should be clear up front that COMPILER-LET is not computationally
essential. Most (if not all) uses of it can be rewritten using MACROLET
or SYMBOL-MACROLET.
A typical use of COMPILER-LET might be:
(defvar *local-type-declarations* '())
(defmacro local-type-declare (declarations &body forms)
`(compiler-let ((*local-type-declarations*
(append ',declarations *local-type-declarations*)))
,@forms))
(defmacro typed-var (var)
(let ((type (assoc var *local-type-declarations*)))
(if type `(the ,(cadr type) ,var) var)))
(defun f (x y)
(local-type-declare ((x fixnum) (y float))
(+ (typed-var x) (typed-var y))))
The same thing could be accomplished using MACROLET:
(defmacro local-type-declare (declarations &body forms)
(local-type-declare-aux declarations forms))
(defmacro typed-var (var) var)
(eval-when (eval compile load)
(defun local-type-declare-aux (declarations forms)
`(macrolet ((typed-var (var)
(let ((type (assoc var ',declarations)))
(if type `(the ,(cadr type) ,var) var)))
(local-type-declare (new-declarations &body new-forms)
(local-type-declare-aux
(append new-declarations ',declarations)
new-forms)))
,@forms)))
A further alternative would be to use SYMBOL-MACROLET (this particular
implementation assumes that issue DEFINING-MACROS-NON-TOP-LEVEL passes):
(let ((temp (gensym)))
(defmacro local-type-declare (declarations &body forms &environment env)
`(symbol-macrolet ((,temp ',(append declarations
(symbol-macro-value temp env))))
,@forms))
(defmacro typed-var (var &environment env)
(let ((type (assoc var (symbol-macro-value temp env))))
(if type `(the ,(cadr type) ,var) var)))
)
(defun symbol-macro-value (symbol env &optional default)
(multiple-value-bind (expansion macro-p) (eval (macroexpand symbol env))
(if macro-p expansion default)))
Opinion is divided as to which is more understandable. Some
people find the COMPILER-LET idiom more understandable, while others
find it just as natural to use MACROLET or SYMBOL-MACROLET.
The issues are these:
- Is it possible to implement COMPILER-LET in a usefully consistent
way in all implementations?
- Are the benefits of providing a useful and compatible implementation
of COMPILER-LET worth any associated cost?
Two proposals are presented below:
- Option REPAIR argues that COMPILER-LET provides interesting
functionality that can be implemented in a manner that is usefully
consistent across implementations, and that the associated cost
is low enough for it to be worthwhile to do so.
- Option ELIMINATE argues that COMPILER-LET complicates the language
and that providing this construct is not worth the associated
implementation cost.
Proposal (COMPILER-LET-CONFUSION:REPAIR):
Strike the existing definition of COMPILER-LET. Redefine it as follows:
COMPILER-LET [Special form]
COMPILER-LET is similar to LET, but it always makes special
bindings and makes those bindings visible only during
macroexpansion of forms in the body, not during the runtime
execution of those forms.
The intent is that some macros might macroexpand into calls to
COMPILER-LET in which the body would the contain references to
macros which access the variables in the COMPILER-LET.
The initial value forms of the bindings, if any, are always
evaluated in a null lexical context, regardless of whether the
COMPILER-LET expression is being interpreted or compiled.
The initial value forms of the bindings, if any, are evaluated in
a dynamic context where the bindings of any lexically enclosing
COMPILER-LET are visible, and where dynamic execution-time
environment may or may not be visible.
Implementation Note: Permitting the execution-time dynamic
environment to be visible when initializing COMPILER-LET variables
is a concession to some interpreters which may have to do this in
order to keep the cost down. Where feasible, implementors should
try not to make the runtime environment visible.
Rationale:
This gives a consistent description of COMPILER-LET which separates
issues of intent from those of implementation in a way that makes it
possible for portable code to make serious use of it, and which does
not force gratuitous incompatibilities between interpreters and
compilers.
This description of COMPILER-LET can be implemented without undue
cost by all implementations. See "Cost to Implementors" for details.
Cost to Implementors:
Modest, but nontrivial in some implementations.
In compiled code, and in interpreters doing a one-time semantic
prepass, it should be fairly easy for COMPILER-LET to cause the
variables to get bound (using PROGV) during semantic analysis.
In interpreters which do not do a semantic-prepass, it is necessary
to fully macroexpand the body. Assuming the presence of a
SYSTEM::MACROEXPAND-ALL primitive, the definition of COMPILER-LET
could look like:
(DEFMACRO COMPILER-LET (BINDINGS &BODY FORMS &ENVIRONMENT ENV)
(SETQ BINDINGS ;; Assure no non-atom bindings
(MAPCAR #'(LAMBDA (BINDING)
(IF (ATOM BINDING) (LIST BINDING) BINDING))
BINDINGS))
(PROGV (MAPCAR #'CAR BINDINGS)
(MAPCAR #'CDR BINDINGS)
(SYSTEM::MACROEXPAND-ALL `(PROGN ,@FORMS) ENV)))
This reduces the problem of writing a program capable of doing a
full macroexpansion. Many systems already have such a facility.
Pitman wrote such a facility in Cloe Runtime in order support
SYMBOL-MACROLET (before it was christened a special form); it was
about 750 lines of relatively straightforward, well-commented code.
Cost to Users:
Code currently depending on this feature is either non-existent or
already not portable (due to wide variation in implementation
strategy for COMPILER-LET).
Most users will probably be happy for any interpretation which offers
them a future shot at portability.
Some users have indicated they dislike interpreters which do a semantic
prepass, because they like to be able to dynamically redefine macros
while debugging.
Proposal (COMPILER-LET-CONFUSION:ELIMINATE):
Remove COMPILER-LET from the language.
Rationale:
Some people think that having one less special form would simplify the
language. The revised COMPILER-LET semantics, which require
COMPILER-LET to make special bindings which are only lexically visible
within its body, are not shared by any other feature in the language,
and require a fairly complex implementation technique. There are
other constructs which are strictly lexical that can be readily used
to solve the same kinds of problems that COMPILER-LET is intended to
be used for.
Cost to Implementors:
Minimal. Implementations could continue to support COMPILER-LET as
an extension.
Cost to Users:
People who use COMPILER-LET would have to rewrite their programs to use
some other construct. As discussed above, most uses of COMPILER-LET
for communication between macros can be handled using MACROLET or
SYMBOL-MACROLET, though some perspicuity may be lost in the process.
Current Practice:
Some implementations have implemented the description in CLtL.
Users of those implementations (quite reasonably) can't figure how to
use COMPILER-LET and so don't use it much.
Some implementations (the ones from which COMPILER-LET originally came)
continue to use their pre-CLtL semantics. These semantics are useful, though
incompatible with CLtL (which they largely consider to simply be in error).
Users of those implementations probably use COMPILER-LET somewhat more
often since it has an intelligible behavior, but their code is not portable
since it relies on behaviors which are either contrary to or not guaranteed
by CLtL.
Benefits:
Either way, a potential area of incompatibility between compiled and
interpreted code would be eliminated.
Either way, a potential area of portability trouble would be very
drastically reduced (in the case of the REPAIR option) or eliminated
(in the case of the ELIMINATE option).
Discussion:
Pitman strongly favors COMPILER-LET-CONFUSION:REPAIR. He argues
against the idea of using MACROLET instead of COMPILER-LET, saying:
This is a little misleading because it's like saying you can
do without LET given that you have FLET. You can, but you lose some things
in the process:
Just as rewriting a LET using FLET might slow your computation, so too
a rewrite of COMPILER-LET using MACROLET might slow things down. However,
compilation speed is generally not weighted as heavily as execution speed
by many people, so the loss of speed here may not be as important.
Just as rewriting a LET using FLET might obscure the simplicity of your
intent, so too rewriting COMPILER-LET using MACROLET might obscure your
intent. You'd probably get used to recognizing idioms if you used it often
enough. Certainly this would be true if you didn't have LET. However,
COMPILER-LET is used less often, so not having it would mean that the
code you wrote instead would be much harder to read because people
wouldn't have the necessary familiarity with the idioms involved and so
wouldn't always understand them.
Sandra Loosemore responds:
The argument that using MACROLET is more inefficient than COMPILER-LET
is questionable. Both of the suggested implementation techniques for
COMPILER-LET involve considerable overhead.
If COMPILER-LET were not part of the language, people wouldn't think in
terms of rewriting COMPILER-LETs as MACROLETs; instead, they'd think of
how to use MACROLET in the first place to solve their problems. This
is what people who now use implementations with broken COMPILER-LETs
already do. Since MACROLET is now used much more frequently than
COMPILER-LET, that argues that people are much more familiar with
MACROLET idioms than COMPILER-LET idioms.
Also, note that the intent of the revised COMPILER-LET definition is
to make the binding only lexically visible within the body. Using
special binding for this purpose is troublesome. Both the MACROLET
and SYMBOL-MACROLET solutions are completely lexical and avoid all
the problems associated with special binding.
-------
∂08-Mar-89 1416 CL-Compiler-mailer issue COMPILER-LET-CONFUSION, version 6
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 8 Mar 89 14:16:17 PST
Received: from BOBOLINK.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 553273; Wed 8-Mar-89 17:13:14 EST
Date: Wed, 8 Mar 89 17:13 EST
From: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>
Subject: issue COMPILER-LET-CONFUSION, version 6
To: sandra%defun@cs.utah.edu
cc: cl-compiler@sail.stanford.edu
In-Reply-To: <8903082159.AA08869@defun.utah.edu>
Message-ID: <890308171306.8.KMP@BOBOLINK.SCRC.Symbolics.COM>
Sigh. I blew it slightly when I suggested that last minute fix to the
SYMBOL-MACROLET rewrite. You have
(defun symbol-macro-value (symbol env &optional default)
(multiple-value-bind (expansion macro-p) (eval (macroexpand symbol env))
(if macro-p expansion default)))
but it should have been
(defun symbol-macro-value (symbol env &optional default)
(multiple-value-bind (expansion macro-p) (macroexpand symbol env)
(if macro-p (eval expansion) default)))
Sorry about that.
∂08-Mar-89 1708 CL-Compiler-mailer A (new) (old) definition of "top-level"
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 8 Mar 89 17:08:36 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 553434; Wed 8-Mar-89 20:05:41 EST
Date: Wed, 8 Mar 89 20:05 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: A (new) (old) definition of "top-level"
To: Sandra J Loosemore <sandra%defun@cs.utah.edu>
cc: cl-compiler@sail.stanford.edu
In-Reply-To: <8903080141.AA08237@defun.utah.edu>
Message-ID: <19890309010519.0.MOON@EUPHRATES.SCRC.Symbolics.COM>
Date: Tue, 7 Mar 89 18:41:35 MST
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
I've spent the past few hours digging around through the literature to
try to get a historical perspective on the use of the term
"top-level". Believe it or not, I've actually found a definition of
the term, from the 1st edition of Winston & Horn's LISP book:
A function is said to be defined or used in the top-level environment
if its definition or use is directly demanded by a user, rather
than indirectly by way of the evaluation of one or more other functions.
I think this is an unrelated use of the same word and doesn't tell
us anything.
Now to to extend this to file compilation. LOADing a source file is
defined as reading and evaluating forms from the file. Since there is
direct evaluation going on, each form that is read from the file is
top-level. This notion applies just as well to COMPILE-FILE, which
must arrange for the loader to "evaluate" or "execute" each of the
top-level forms.
Finally, we have implicitly accepted the idea that macroexpansion by
the compiler must preserve semantics. That implies that
macroexpansion should not have any effect on top-level-ness. Thinking
about our simple compiler model that doesn't do much besides walk the
code to expand all macros and print out the results, if it is allowed
to turn the bodies of COMPILER-LETs, MACROLETs, and SYMBOL-MACROLETs
into PROGNs, then that would be an argument for saying that those
three special forms should be treated the same as PROGN as far as
top-level-ness goes. However, that rationale wouldn't cover
constructs like LOCALLY and LABELS (as Moon suggests) unless we modify
our compiler model.
This way of thinking about top-level-ness makes sense to me. How
about the rest of you?
I think this makes sense, but all it says is that a top-level form is
one that is not inside another form.
We need to go back and ask why the compiler has to have a concept of
top-level form in the first place. (Surely it would be better if we
could just jettison the concept entirely.) I think the concept exists
only because in compile-file defmacro as a top-level form does something
rather different from defmacro as a non-top-level form. There are a few
others (deftype, defsetf) in the same category. Then the important
thing about top-level forms is not whether they are enclosed in
parentheses, but whether they can get this special compiler treatment of
defmacro. I could now repeat what I said yesterday but I'll spare you.
I think it fits very well with your simple compiler model, though.
∂08-Mar-89 1719 CL-Compiler-mailer Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 8 Mar 89 17:19:21 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 553441; Wed 8-Mar-89 20:16:02 EST
Date: Wed, 8 Mar 89 20:15 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Re: draft of alternate proposal for EVAL-WHEN-NON-TOP-LEVEL
To: Sandra J Loosemore <sandra%defun@cs.utah.edu>
cc: Eric Benson <eb@lucid.com>, pierson@mist.encore.com, cl-compiler@sail.stanford.edu
In-Reply-To: <8903072255.AA08178@defun.utah.edu>
Message-ID: <19890309011538.1.MOON@EUPHRATES.SCRC.Symbolics.COM>
Date: Tue, 7 Mar 89 15:55:10 MST
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
> Date: Tue, 7 Mar 89 17:18 EST
> From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
>
> It makes me wonder if
> perhaps we shouldn't retreat to a minimalist position where PROGN is
> the only special case.
>
> Do we really have to rehearse all the arguments as to why that won't work
> once again?
I'm not aware that such arguments have ever been coherently stated. I
just went through all my back mail on this subject and couldn't find
anything on it from you or from anyone else.
I think you're right that it would work to not make locally, macrolet,
labels, and so forth when used at top-level treat their bodies as
top-level. It would be less esthetic, but it wouldn't fail to work.
I was confused on that point. What I was thinking of was eval-when;
CLtL p.70 seems quite clear that when an eval-when is processed
as a top-level form, and the situation LOAD is specified, the forms
in the body are also processed as top-level forms. It doesn't use
the word "top-level" but from the context that's what "process" means.
I also really believe that it will not work to change this so that
the body of an eval-when is not top-level.
Perhaps we need to be reminded that the minimalist position is
actually the status quo -- PROGN is the only special case mentioned by
CLtL.
I'd claim PROGN and EVAL-WHEN. I agree that CLtL doesn't require anything
else.
I question whether the problem with LOCALLY justifies trying to come
up with a model of what toplevelness is that is radically different
than what is specified in CLtL.
No, what justifies coming up with a model is that noone can understand
the description in CLtL, including I think its authors.
∂09-Mar-89 0832 CL-Compiler-mailer Re: issue COMPILER-LET-CONFUSION, version 6
Received: from NSS.Cs.Ucl.AC.UK by SAIL.Stanford.EDU with TCP; 9 Mar 89 08:30:56 PST
Received: from aiai.edinburgh.ac.uk by NSS.Cs.Ucl.AC.UK via Janet with NIFTP
id aa07305; 9 Mar 89 15:31 GMT
Date: Thu, 9 Mar 89 16:00:42 GMT
Message-Id: <29341.8903091600@subnode.aiai.ed.ac.uk>
From: Jeff Dalton <jeff%aiai.edinburgh.ac.uk@NSS.Cs.Ucl.AC.UK>
Subject: Re: issue COMPILER-LET-CONFUSION, version 6
To: sandra (Sandra J Loosemore) <@cs.utah.edu:sandra@defun>,
cl-compiler@sail.stanford.edu
In-Reply-To: Sandra J Loosemore's message of Wed, 8 Mar 89 14:59:05 MST
> Problem Description:
>
> The description of the COMPILER-LET special form in CLtL is confusing
> to many people. There are no examples provided to make it clear how it
> is supposed to be used. The only description which is offered is overly
> concrete, which have led to confusion about the intent of COMPILER-LET,
↑↑↑↑ has
> and about its implementability.
>
> The intent of COMPILER-LET was to permit information to be communicated
> between macros by use of dynamic variables at macroexpansion time.
> It was not necessary to the intended uses of COMPILER-LET that such
> variables ever be bound at execution time.
>
> Unfortunately, probably because some implementations did not primitively
> support COMPILER-LET at the time CLtL was written, an exception was
> permitted to make COMPILER-LET `more or less work' in interpreters:
Is it really "interpreters" (verses some unspecified other thing), or
macroexpansion time verses executiontime?
> the COMPILER-LET variables were permitted to be bound at execution time.
> The problem was further compounded by the fact that CLtL presented this
> exception as part of COMPILER-LET's contract rather than as an
> implementation note, and by the fact that no examples of actually using
> COMPILER-LET correctly are provided.
>
> Subtle bugs can be introduced because of the different handling of the
> variable bindings in the interpreter and the compiler. In compiled
> code, the bindings are only lexically visible during the expansion of
> macros at compile time, while in interpreted code the bindings have
> dynamic scope and may also be seen during ordinary evaluation if
> evaluation and macroexpansion happen concurrently.
Concurrently?
> Further compatibility problems can result from the value forms being
> evaluated in a null lexical environment in the compiler and the ordinary
> lexical environment in the interpreter.
There's nothing here about occasions such as
(compiler-let ((*v* 1))
#'(lambda () (m)))
where M is a macro that refers to *V*, but the lambda-expr isn't
macroexpanded until it's finally called, and so parhpas after the
dynamic extent of the *V* binding has ended. Isn't this part of
the "problem"?
> Background and Analysis:
> [...]
> Opinion is divided as to which is more understandable. Some
> people find the COMPILER-LET idiom more understandable, while others
> find it just as natural to use MACROLET or SYMBOL-MACROLET.
There's also the problem that the COMPILER-LET may not work, due
to the extent problem noted above.
> Proposal (COMPILER-LET-CONFUSION:REPAIR):
> [...]
> In interpreters which do not do a semantic-prepass, it is necessary
> to fully macroexpand the body. Assuming the presence of a
> SYSTEM::MACROEXPAND-ALL primitive, the definition of COMPILER-LET
> could look like:
> (DEFMACRO COMPILER-LET (BINDINGS &BODY FORMS &ENVIRONMENT ENV)
> (SETQ BINDINGS ;; Assure no non-atom bindings
> (MAPCAR #'(LAMBDA (BINDING)
> (IF (ATOM BINDING) (LIST BINDING) BINDING))
> BINDINGS))
> (PROGV (MAPCAR #'CAR BINDINGS)
> (MAPCAR #'CDR BINDINGS)
> (SYSTEM::MACROEXPAND-ALL `(PROGN ,@FORMS) ENV)))
> This reduces the problem of writing a program capable of doing a
> full macroexpansion. Many systems already have such a facility.
> Pitman wrote such a facility in Cloe Runtime in order support
> SYMBOL-MACROLET (before it was christened a special form); it was
> about 750 lines of relatively straightforward, well-commented code.
The need for a prepass (why "semantic prepass"?) isn't explained,
and there's nothing in the problem description (about the extent issue,
say) that indicates there's a problem the prepass solves.
> Cost to Users:
>
> Code currently depending on this feature is either non-existent or
> already not portable (due to wide variation in implementation
> strategy for COMPILER-LET).
>
> Most users will probably be happy for any interpretation which offers
> them a future shot at portability.
>
> Some users have indicated they dislike interpreters which do a semantic
> prepass, because they like to be able to dynamically redefine macros
> while debugging.
Can macros be expanded to something that records the COMPILER-LET
nevironment so that they could be correctly re-expanded later? For
example, suppose macro expansions were cached (in a hash table, say)
and the original code left in place. Then, if the macro is redefined,
the expansion is recomputed. The C-LET environment could be saved
along with the expansion. Would that work?
> Proposal (COMPILER-LET-CONFUSION:ELIMINATE):
>
> Remove COMPILER-LET from the language.
>
> Rationale:
>
> Some people think that having one less special form would simplify the
> language. The revised COMPILER-LET semantics, which require
> COMPILER-LET to make special bindings which are only lexically visible
"only lexically visible" isn't quite right. It sounds like you're
saying they have lexical scope, but they don't because they're visible
to macro functions. So I think they have indefinite scope and dynamic
extent, like all special variables, but the extent is macroexpansion
time rather than execution time.
> within its body, are not shared by any other feature in the language,
> and require a fairly complex implementation technique. There are
> other constructs which are strictly lexical that can be readily used
> to solve the same kinds of problems that COMPILER-LET is intended to
> be used for.
> [...]
> Current Practice:
>
> Some implementations have implemented the description in CLtL.
> Users of those implementations (quite reasonably) can't figure how to
> use COMPILER-LET and so don't use it much.
>
> Some implementations (the ones from which COMPILER-LET originally came)
> continue to use their pre-CLtL semantics. These semantics are useful, though
> incompatible with CLtL (which they largely consider to simply be in error).
I always find this mysterious because I don't know what the "pre CLtL"
semantics was (if it's diffeerent).
> Users of those implementations probably use COMPILER-LET somewhat more
> often since it has an intelligible behavior, but their code is not portable
> since it relies on behaviors which are either contrary to or not guaranteed
> by CLtL.
> [...]
-- Jeff
∂09-Mar-89 1102 CL-Compiler-mailer Issue MACRO-ENVIRONMENT-EXTENT
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 9 Mar 89 11:02:43 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 553814; Thu 9-Mar-89 13:59:24 EST
Date: Thu, 9 Mar 89 13:59 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Issue MACRO-ENVIRONMENT-EXTENT
To: Kim A. Barrett <IIM%ECLA@ECLC.USC.EDU>, sandra%defun@CS.UTAH.EDU
cc: cl-compiler@SAIL.STANFORD.EDU, common-lisp-object-system@SAIL.STANFORD.EDU
In-Reply-To: <12472024104.5.IIM@ECLA.USC.EDU>
Message-ID: <19890309185905.3.MOON@EUPHRATES.SCRC.Symbolics.COM>
Date: Sun 19 Feb 89 15:54:36-PST
From: Kim A. Barrett <IIM%ECLA@ECLC.USC.EDU>
The extent of macro environment objects is related to EVAL-WHEN because macro
expanders may wish to return forms which contain environments as quoted
constants.
I am convinced that this should be ruled out, and that CLOS made a mistake
here. (Incidentally the part of CLOS that says this is in chapter 3, the
accepted part of CLOS does not say anything about the expansion of the
macros is.)
....
Requiring environments to have indefinite extent has
problems for CLOS because at compile-time it wants to create remote metaobjects
and link them into the right places, but then flush those links when the
compilation is over.
This depends on whether you think the environment actually contains the
table that relates names to objects, or just contains a boolean flag
that tells functions such as FIND-CLASS which of two tables to look in.
Under the latter model, nothing about the environment prevents the
COMPILE-FILE table from being reset at any time. This is one reason
why I think the second model is right.
As far as MACRO-ENVIRONMENT-EXTENT itself goes, I am convinced it should
be dynamic extent. I'm also convinced that the decision on this issue
does not affect CLOS.
∂09-Mar-89 1301 Common-Lisp-Object-System-mailer Re: Issue: LOAD-OBJECTS (Version 2)
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 9 Mar 89 13:01:40 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 554005; Thu 9-Mar-89 15:58:57 EST
Date: Thu, 9 Mar 89 15:58 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Re: Issue: LOAD-OBJECTS (Version 2)
To: David N Gray <Gray@DSG.csc.ti.com>
cc: Common-Lisp-Object-System@SAIL.STANFORD.EDU, CL-Compiler@SAIL.STANFORD.EDU
In-Reply-To: <2809731258-5200907@Kelvin>
Message-ID: <19890309205844.0.MOON@EUPHRATES.SCRC.Symbolics.COM>
Date: Fri, 13 Jan 89 18:54:18 CST
From: David N Gray <Gray@DSG.csc.ti.com>
This looks good. The only thing I have doubts about is:
> The function MAKE-LOAD-FORM-USING-SLOTS can be useful in user-written
> MAKE-LOAD-FORM methods. Its first argument is the object. Its
> optional second argument is a list of the names of the slots to
> preserve; it defaults to all of the local slots.
> MAKE-LOAD-FORM-USING-SLOTS returns forms that construct an equivalent
> object using MAKE-INSTANCE and SETF of SLOT-VALUE for slots with
> values, or SLOT-MAKUNBOUND for slots without values, or using other
> functions of equivalent effect.
Rather than having the second argument default to a list of all instance
slots, it might be better to consider two separate cases:
1. If a second argument is supplied, then MAKE-INSTANCE will be used to
create the object, (using INITIALIZE-INSTANCE to default the slot
values), and then the designated slots will be forced to
have the proper value.
2. Without a second argument, ALLOCATE-INSTANCE will be used to create
the object (without invoking INITIALIZE-INSTANCE or
SHARED-INITIALIZE), and then all the slots will be filled in.
If you are going to specify all of the slot values, then there shouldn't
be a need to compute default values, and it may be undesirable to invoke
INITIALIZE-INSTANCE -- for example, it might complain about missing
required arguments or perform undesired side-effects.
I don't think it's a good idea to have such a large deviation in behavior
based on whether an optional argument is present or not. What if the
argument is present but its value is a list of all the slots?
I personally cannot figure out whether calling INITIALIZE-INSTANCE when
it's not wanted, or failing to call it when it is wanted, would cause
more unexpected behavior. I have to resolve that by keeping it simple
so the programmer can figure it out on his own. So I think it should
always create the object with MAKE-INSTANCE.
> The default MAKE-LOAD-FORM method for STANDARD-OBJECT signals an
> error.
Wouldn't it be permissible to just not have a default method, so that a
"no applicable method" error is signalled?
Agreed.
∂09-Mar-89 1334 CL-Cleanup-mailer issue IN-PACKAGE-FUNCTIONALITY, version 6
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 9 Mar 89 13:34:10 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 554064; Thu 9-Mar-89 16:31:08 EST
Date: Thu, 9 Mar 89 16:30 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: issue IN-PACKAGE-FUNCTIONALITY, version 6
To: Sandra J Loosemore <sandra%defun@cs.utah.edu>
cc: cl-cleanup@sail.stanford.edu, cl-compiler@sail.stanford.edu
In-Reply-To: <8901310224.AA25846@defun.utah.edu>
Message-ID: <19890309213057.3.MOON@EUPHRATES.SCRC.Symbolics.COM>
IN-PACKAGE-FUNCTIONALITY:NEW-MACRO is fine with me. Typo:
the name of the new macro is misspelled in the example code
in the cost to implementors section.
I don't like the alternate IN-PACKAGE-FUNCTIONALITY:SELECT-ONLY
proposal as well.
∂09-Mar-89 1329 CL-Cleanup-mailer Issue: LOAD-OBJECTS (Version 3)
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 9 Mar 89 13:29:07 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 554052; Thu 9-Mar-89 16:26:35 EST
Date: Thu, 9 Mar 89 16:26 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Issue: LOAD-OBJECTS (Version 3)
To: CL-Cleanup@sail.stanford.edu, CL-Compiler@sail.stanford.edu, Common-Lisp-Object-System@sail.stanford.edu
Message-ID: <19890309212623.2.MOON@EUPHRATES.SCRC.Symbolics.COM>
At Kauai I was asked to keep working on this and come up with a modified
version based on comments received. Here it is. I hope this is ready
for voting so we can clear it out of the way.
Issue: LOAD-OBJECTS
References: none
Related issues: LOAD-TIME-EVAL,
CONSTANT-COMPILABLE-TYPES,
CONSTANT-CIRCULAR-COMPILATION
Category: ADDITION
Forum: Cleanup
Edit history: Version 1, 2-Jan-89, by Moon (for discussion)
Version 2, 13-Jan-89, by Moon (draft updated from discussion)
Version 3, 9-Mar-89, by Moon (changes suggested by discussion)
Problem description:
Common Lisp doesn't provide any way to use an object of a user-defined
type (defined with DEFCLASS or DEFSTRUCT) as a constant in a program
compiled with COMPILE-FILE. The problem is that LOAD has to be able
to "reconstruct" an equivalent object when the compiled-code file is
loaded, but the programmer has no way to tell LOAD how to do that.
Proposal (LOAD-OBJECTS:MAKE-LOAD-FORM):
Define a new generic function named MAKE-LOAD-FORM, which takes one
argument and returns two values. The argument is an object that is
referenced as a constant or as a self-evaluating form in a file being
compiled by COMPILE-FILE. The objective is to enable LOAD to
construct an equivalent object.
The first value, called the "creation form," is a form that, when
evaluated at load time, should return an object that is equivalent to
the argument. The exact meaning of "equivalent" depends on the type
of object and is up to the programmer who defines a method for
MAKE-LOAD-FORM. This is the same type of equivalence discussed
in issue CONSTANT-COMPILABLE-TYPES.
The second value, called the "initialization form," is a form that,
when evaluated at load time, should perform further initialization of
the object. The value returned by the initialization form is ignored.
If the MAKE-LOAD-FORM method returns only one value, the
initialization form is NIL, which has no effect. If the object used
as the argument to MAKE-LOAD-FORM appears as a constant in the
initialization form, at load time it will be replaced by the
equivalent object constructed by the creation form; this is how the
further initialization gains access to the object.
Both the creation form and the initialization form can contain
references to objects of user-defined types (defined precisely below).
However, there must not be any circular dependencies in creation forms.
An example of a circular dependency is when the creation form for the
object X contains a reference to the object Y, and the creation form
for the object Y contains a reference to the object X. A simpler
example would be when the creation form for the object X contains
a reference to X itself. Initialization forms are not subject to
any restriction against circular dependencies, which is the entire
reason that initialization forms exist. See the example of circular
data structures below.
The creation form for an object is always evaluated before the
initialization form for that object. When either the creation form or
the initialization form references other objects of user-defined types
that have not been referenced earlier in the COMPILE-FILE, the
compiler collects all of the creation and initialization forms. Each
initialization form is evaluated as soon as possible after its
creation form, as determined by data flow. If the initialization form
for an object does not reference any other objects of user-defined
types that have not been referenced earlier in the COMPILE-FILE, the
initialization form is evaluated immediately after the creation form.
If a creation or initialization form F references other objects of
user-defined types that have not been referenced earlier in the
COMPILE-FILE, the creation forms for those other objects are evaluated
before F, and the initialization forms for those other objects are
also evaluated before F whenever they do not depend on the object
created or initialized by F. Where the above rules do not uniquely
determine an order of evaluation, which of the possible orders of
evaluation is chosen is unspecified.
While these creation and initialization forms are being evaluated, the
objects are possibly in an uninitialized state, analogous to the state
of an object between the time it has been created by ALLOCATE-INSTANCE
and it has been processed fully by INITIALIZE-INSTANCE. Programmers
writing methods for MAKE-LOAD-FORM must take care in manipulating
objects not to depend on slots that have not yet been initialized.
It is unspecified whether LOAD calls EVAL on the forms or does some
other operation that has an equivalent effect. For example, the
forms might be translated into different but equivalent forms and
then evaluated, they might be compiled and the resulting functions
called by LOAD, or they might be interpreted by a special-purpose
interpreter different from EVAL. All that is required is that the
effect be equivalent to evaluating the forms.
COMPILE-FILE calls MAKE-LOAD-FORM on any object that is referenced as
a constant or as a self-evaluating form, if the object's metaclass is
STANDARD-CLASS, STRUCTURE-CLASS, any user-defined metaclass (not a
subclass of BUILT-IN-CLASS), or any of a possibly-empty
implementation-defined list of other metaclasses. COMPILE-FILE will
only call MAKE-LOAD-FORM once for any given object (compared with EQ)
within a single file.
It is valid for user programs to call MAKE-LOAD-FORM in other
circumstances, providing the argument's metaclass is not BUILT-IN-CLASS
or a subclass of BUILT-IN-CLASS.
Define a new function named MAKE-LOAD-FORM-USING-SLOTS, which takes
one required argument and one optional argument and returns two
values. This can be useful in user-written MAKE-LOAD-FORM methods.
The first argument is the object. The optional second argument is a
list of the names of the slots to preserve; it defaults to all of the
local slots. MAKE-LOAD-FORM-USING-SLOTS returns forms that construct
an equivalent object using MAKE-INSTANCE and SETF of SLOT-VALUE for
slots with values, or SLOT-MAKUNBOUND for slots without values, or
using other functions of equivalent effect.
MAKE-LOAD-FORM-USING-SLOTS returns two values, thus it can deal with
circular structures. MAKE-LOAD-FORM-USING-SLOTS works for any object
of metaclass STANDARD-CLASS or STRUCTURE-CLASS. Whether the result is
useful in an application depends on whether the object's type and slot
contents fully capture the application's idea of the object's state.
MAKE-LOAD-FORM of an object of metaclass STANDARD-CLASS or
STRUCTURE-CLASS for which no user-defined method is applicable signals
an error. It is valid to implement this either by defining default
methods on STANDARD-OBJECT and STRUCTURE-OBJECT that signal an error
or by having no applicable method for those classes.
Examples:
;; Example 1
(defclass my-class ()
((a :initarg :a :reader my-a)
(b :initarg :b :reader my-b)
(c :accessor my-c)))
(defmethod shared-initialize ((self my-class) ignore &rest ignore)
(unless (slot-boundp self 'c)
(setf (my-c self) (some-computation (my-a self) (my-b self)))))
(defmethod make-load-form ((self my-class))
`(make-instance ',(class-name (class-of self))
:a ',(my-a self) :b ',(my-b self)))
In this example, an equivalent instance of my-class is reconstructed
by using the values of two of its slots. The value of the third slot
is derived from those two values.
Another way to write the last form in the above example would have been
(defmethod make-load-form ((self my-class))
(make-load-form-using-slots self '(a b)))
;; Example 2
(defclass my-frob ()
((name :initarg :name :reader my-name)))
(defmethod make-load-form ((self my-frob))
`(find-my-frob ',(my-name self) :if-does-not-exist :create))
In this example, instances of my-frob are "interned" in some way.
An equivalent instance is reconstructed by using the value of the
name slot as a key for searching existing objects. In this case
the programmer has chosen to create a new object if no existing
object is found; alternatively she could have chosen to signal an
error in that case.
;; Example 3
(defclass tree-with-parent () ((parent :accessor tree-parent)
(children :initarg :children)))
(defmethod make-load-form ((x tree-with-parent))
(values
;; creation form
`(make-instance ',(class-of x) :children ',(slot-value x 'children))
;; initialization form
`(setf (tree-parent ',x) ',(slot-value x 'parent))))
In this example, the data structure to be dumped is circular, because
each parent has a list of its children and each child has a reference
back to its parent. Suppose make-load-form is called on one object in
such a structure. The creation form creates an equivalent object and
fills in the children slot, which forces creation of equivalent
objects for all of its children, grandchildren, etc. At this point
none of the parent slots have been filled in. The initialization form
fills in the parent slot, which forces creation of an equivalent
object for the parent if it was not already created. Thus the entire
tree is recreated at load time. At compile time, MAKE-LOAD-FORM is
called once for each object in the true. All of the creation forms
are evaluated, in unspecified order, and then all of the
initialization forms are evaluated, also in unspecified order.
;; Example 4
(defstruct my-struct a b c)
(defmethod make-load-form ((s my-struct))
(make-load-form-using-slots s))
In this example, the data structure to be dumped has no special
properties and an equivalent structure can be reconstructed
simply by reconstructing the slots' contents.
Rationale:
Only the programmer who designed a class can know the correct
way to reconstruct objects of that class at load time, therefore
the reconstruction should be controlled by a generic function.
Using EVAL as the interface for telling LOAD what to do provides
full generality.
MAKE-LOAD-FORM returns two values so that circular structures can
be handled. If CONSTANT-CIRCULAR-COMPILATION is rejected,
MAKE-LOAD-FORM will only return one value, although implementations
that make an extension to support circular constants will probably
also make the extension to accept two values from MAKE-LOAD-FORM.
The default for class objects and structures is to signal an error,
rather than picking some particular object reconstruction technique,
because no reconstruction technique is appropriate for all objects.
It only takes two lines of code, as in example 4, to instruct the
compiler to use the technique that most often has been suggested
as the default.
MAKE-LOAD-FORM has a natural resemblance to PRINT-OBJECT, as a hook
for the programmer to control the system's actions.
The order of evaluation rules for creation and initialization forms
eliminate the possibility of partially initialized objects in the
absence of circular structures, and reduce it to the minimum possible
in the presence of circular structures. This allows nodes in
non-circular structures to be built out of fully initialized subparts.
Current practice:
Symbolics Flavors has something like this, but under a different name.
The name Symbolics uses is not suitable for standardization.
JonL reports that Lucid is getting more and more requests for this.
Cost to Implementors:
This seems like only a few one-line changes in the compiled-code
file writer and reader. MAKE-LOAD-FORM-USING-SLOTS is a couple
dozen lines of code, assuming the presence of the CLOS metaobject
protocol or an implementation-dependent equivalent.
Cost to Users:
None.
Cost of non-adoption:
Serious impairment of the ability to use extended-type objects. Each
implementation will probably make up its own version of this as an
extension.
Performance impact:
None.
Benefits:
See Cost of non-adoption.
Esthetics:
No significant positive or negative impact.
Discussion:
It would be possible to define an additional level of protocol that
allows multiple classes to contribute to the reconstruction of an
object, combining initialization arguments contributed by each class.
Since a user can easily define that in terms of MAKE-LOAD-FORM without
modifying the Lisp system, it is not being proposed now.
Any type that has a read syntax is likely to appear as a quoted
constant or inside a quoted constant. Pathnames are one example, user
programs often define others. Also many implementations provide a way
to create a compiled-code file full of data (rather than compiled Lisp
programs), and such data probably include extended-type objects.
Moon supports this. David Gray and John Rose made major contributions
to the discussion that produced this improved version 2 proposal.
∂09-Mar-89 1325 CL-Cleanup-mailer Issue: LOAD-OBJECTS (Version 2)
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 9 Mar 89 13:25:46 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 554041; Thu 9-Mar-89 16:22:52 EST
Date: Thu, 9 Mar 89 16:22 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Issue: LOAD-OBJECTS (Version 2)
To: John Rose <jrose@Sun.COM>
cc: CL-Cleanup@SAIL.STANFORD.EDU, Common-Lisp-Object-System@SAIL.STANFORD.EDU,
CL-Compiler@SAIL.STANFORD.EDU
In-Reply-To: <8901140458.AA18401@lukasiewicz.sun.com>
Message-ID: <19890309212238.1.MOON@EUPHRATES.SCRC.Symbolics.COM>
Date: Fri, 13 Jan 89 20:58:49 PST
From: jrose@Sun.COM (John Rose)
...
The creation form for an object is always evaluated before the
initialization form for that object. When either the creation form or
the initialization form references other objects of user-defined types
that have not been referenced earlier in the COMPILE-FILE, the
compiler collects all of the creation forms together and collects all
of the initialization forms together. All of the creation forms are
evaluated before any of the initialization forms. The order of
evaluation of the creation forms is unspecified except when the
ordering is forced by data dependencies. The order of evaluation of
the initialization forms is unspecified.
...
Why does the proposal restrict the evaluation initialization forms to
such a late time? Data dependencies would allow an object X's
initialization form to be executed any time after X's creation form had
finished.
Actually, it would be better (and no more difficult, it seems to me) to
be strict in the other direction: Objects should be initialized as early
as possible, and hence at a deterministic time. This would allow nodes
in non-circular structures to be built out of fully initialized subparts,
which is clearly something an application could need.
Good point. I've modified the proposal accordingly, although I did not use
your exact wording. Of course the time is not fully determinstic, but
it's more deterministic than in version 2 of the proposal.
∂09-Mar-89 1421 CL-Compiler-mailer issue MACRO-ENVIRONMENT-EXTENT, version 2
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 9 Mar 89 14:21:02 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA01983; Thu, 9 Mar 89 15:18:38 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA09917; Thu, 9 Mar 89 15:18:34 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903092218.AA09917@defun.utah.edu>
Date: Thu, 9 Mar 89 15:18:33 MST
Subject: issue MACRO-ENVIRONMENT-EXTENT, version 2
To: cl-compiler@sail.stanford.edu
This writeup ended up getting more changes than I anticipated. Here it
is in case anybody has any additional comments before I send it out
to X3J13.
Forum: Compiler
Issue: MACRO-ENVIRONMENT-EXTENT
References: CLtL p. 145-146
Issue COMPILER-LET-CONFUSION
Issue MACRO-CACHING
Issue EVAL-WHEN-NON-TOP-LEVEL
Issue SYNTACTIC-ENVIRONMENT-ACCESS
CLOS Chapter 3 (89-003)
Category: CLARIFICATION,CHANGE
Edit History: V1, 10 Jan 1989, Sandra Loosemore
V2, 09 Mar 1989, Sandra Loosemore
Status: Ready for release
Problem Description:
What is the extent of environment objects received as the &ENVIRONMENT
argument of a macro function?
CLtL says that &ENVIRONMENT is "useful primarily in the rare cases
where a macro definition must explicitly expand any macros in a
subform of the macro call before computing its own expansion". While
this suggests that macro environment objects are typically used within
the dynamic scope of the macro function, the use of the word
"primarily" (rather than "only" or "exclusively" or some similarly
strong language) suggests that there may be other legitimate uses for
environment objects. But, because CLtL is silent about what those
uses might be, many users and implementors are under the impression
that environment objects have only dynamic extent.
There are some situations where using environment objects as if they
had indefinite extent provides a very useful viewpoint from which to
solve a problem. Consider the following example:
(defmacro typed-var (var) var)
(defmacro local-type-declare (declarations &body forms &environment env)
`(macrolet ((typed-var (&whole w var)
(let ((type (assoc var ',declarations)))
(if type
`(the ,(cadr type) ,var)
(macroexpand w ',env)))))
,@forms))
(defun f (x y)
(local-type-declare ((x fixnum) (y float))
(+ (typed-var x) (typed-var y))))
Here, local macro TYPED-VAR is defined to look first in the innermost
lexical environment for information about the variable, and if there
isn't any then it recurses on the next outermost lexical environment.
The global definition of TYPED-VAR provides a terminal case to stop
the recursion.
There are other situations where the extent of macro environment
objects comes into play. For example, if we allow caching of macro
expansions (issue MACRO-CACHING), environments must have indefinite
extent. It is unclear whether CLOS would be affected by allowing
macro environments to have only dynamic extent. (The descriptions of
the CLOS defining macros in document 89-003 seem to imply that the
value of the &ENVIRONMENT argument appears in the expansion of the
macro, but there now seems to be sentiment that the model of how the
defining macros work that is presented there is broken.)
Proposal MACRO-ENVIRONMENT-EXTENT:INDEFINITE:
State that macro environment objects received with the &ENVIRONMENT
argument of a macro function or as the argument to a *MACROEXPAND-HOOK*
function have indefinite extent.
Note that implementations are not permitted to destructively modify
environment objects once they have been passed to a macro function.
Rationale:
This legitimizes the use of idioms which depend on macro environments
having indefinite extent.
Since data objects in Lisp otherwise have indefinite extent, it is
more consistent to give environment objects indefinite extent as
well.
Proposal MACRO-ENVIRONMENT-EXTENT:DYNAMIC:
State that macro environment objects received with the &ENVIRONMENT
argument of a macro function or with a *MACROEXPAND-HOOK* function
have only dynamic extent; the consequences are undefined if they are
referred to outside the dynamic extent of that macro function or hook
function.
Rationale:
This allows implementations to use somewhat more efficient techniques
for representing environment objects. For example, the storage could
be stack-allocated, or environments could be bashed destructively
instead of always being freshly heap-allocated.
Proposal MACRO-ENVIRONMENT-EXTENT:DYNAMIC-WITH-COPIER:
State that macro environment objects received with the &ENVIRONMENT
argument of a macro function or with a *MACROEXPAND-HOOK* function
have only dynamic extent; the consequences are undefined if they are
referred to outside the dynamic extent of that macro function or hook
function.
Add a new function:
COPY-ENVIRONMENT environment [function]
This function returns an environment object that is semantically
equivalent to "environment" (which must be an object of the type
received with an &ENVIRONMENT argument to a macro or as an argument to
a *MACROEXPAND-HOOK* function), but which may safely be referred to
outside the dynamic extent of the macro function. This function is
permitted to return an object that is EQ to its argument if that
object may be safely used.
Rationale:
This allows implementations to use somewhat more efficient techniques
for representing environment objects. For example, the storage could
be stack-allocated, or environments could be bashed destructively
instead of always being freshly heap-allocated.
It also allows programmers to use idioms that rely on environment
objects having indefinite extent.
Current Practice:
Macro environments appear to have indefinite extent in Lucid Common
Lisp, Kyoto Common Lisp, CMU Common Lisp (at least in the
interpreter), Utah Common Lisp, and A-Lisp. A-Lisp internally uses
the kind of idiom shown in the example above to implement FLET,
LABELS, and FUNCTION as macros.
Macro environments are stack-allocated in Symbolics Genera.
Cost to implementors:
For proposal INDEFINITE, some implementations may need to change. A
simple implementation of macro environments that would fit the
requirements of this proposal is to represent them as lists, pushing
information for inner contours onto the front of the list as the
contour is entered and popping the list when the contour is exited.
Implementations that now use some other representation, or that
stack-allocate environments, could make a copy of the environment
before expanding any macros.
For proposal DYNAMIC, there is no associated implementation cost.
For proposal DYNAMIC-WITH-COPIER, the implementation cost is unknown
but probably trivial in most implementations.
Cost to users:
For proposal INDEFINITE, there is no associated cost to users.
For proposal DYNAMIC, users would not be able to portably use a
simple and elegant approach to solving certain kinds of problems.
For proposal DYNAMIC-WITH-COPIER, users would have to remember to make
a copy of an environment object in some situations.
Benefits:
It is made clear whether treating environment objects as if they had
indefinite extent is portable usage.
Discussion:
Proposal SYNTACTIC-ENVIRONMENT-ACCESS:ADD-FUNCTIONAL-INTERFACE
includes adding a function called AUGMENT-ENVIRONMENT. It's unclear
whether having this function would eliminate the need for
COPY-ENVIRONMENT under proposal DYNAMIC-WITH-COPIER.
Loosemore supports proposal MACRO-ENVIRONMENT-EXTENT:INDEFINITE.
Moon says:
My opinion is that anything in CLOS that seems to depend on indefinite
extent for macro environments is broken and needs to be fixed. It's not
broken because of the environment extent, but for other reasons.
Thus I believe in dynamic extent for environments.
Neil Goldman says:
In my code walker I have a pretty ugly way of dealing with MACROLET
that would have been trivial if I could have counted on the
ENVIRONMENT having indefinite extent. There may be some cleaner way
than what I did, but I just looked at PCL's code walker, and it also
is much more complex than would be necessary if environments had
indefinite extent.
-------
∂09-Mar-89 1539 CL-Compiler-mailer Issue: LOCALLY-TOP-LEVEL (Version 1)
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 9 Mar 89 15:38:57 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 554236; Thu 9-Mar-89 18:36:20 EST
Date: Thu, 9 Mar 89 18:36 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Issue: LOCALLY-TOP-LEVEL (Version 1)
To: CL-Cleanup@sail.stanford.edu
cc: Sandra J Loosemore <sandra%defun@cs.utah.edu>, CL-Compiler@sail.stanford.edu
In-Reply-To: <8903091815.AA09753@defun.utah.edu>
Message-ID: <19890309233609.6.MOON@EUPHRATES.SCRC.Symbolics.COM>
This is a cleanup issue, but I'm cc'ing the compiler committee
for their information.
Issue: LOCALLY-TOP-LEVEL
References: None
Related issues: EVAL-WHEN-NON-TOP-LEVEL, DECLARATION-SCOPE
Category: CLARIFICATION / ADDITION
Edit history: Version 1, 9-Mar-89, by Moon
Problem description:
It is desirable to be able to wrap LOCALLY around one or more
top-level forms and have them continue to be treated as top-level
forms. Three examples of how this is useful:
- to put an OPTIMIZE or INLINE declaration into force around
several related forms.
- to put declarations into force around DEFCLASS, or any other
top-level form that lacks a syntax for embedded declarations.
- DECLARATION-SCOPE:LIMITED-HOISTING, which passed in January,
removed the ability to use a DECLARE at the head of the body of a
DEFUN or DEFMACRO to make a declaration that applies to the entire
form, including the lambda-list. We are supposed to use LOCALLY
instead, but forms in the body of LOCALLY are not top-level,
and that changes the semantics of DEFMACRO.
Issue EVAL-WHEN-NON-TOP-LEVEL could not define LOCALLY to treat
its body as top-level forms, because only a special form can do
that and LOCALLY is a macro.
Proposal (LOCALLY-TOP-LEVEL:SPECIAL-FORM):
Change LOCALLY from a macro to a special form, and change the
definition of compiler processing (in EVAL-WHEN-NON-TOP-LEVEL)
so that when a LOCALLY form appears at top level the forms in
its body are processed at top level.
Examples:
(locally (declare (optimize (safety 3) (space 3) (speed 0)))
(defmacro frob (&environment e x y &optional (z (foo x y)))
(mumble x y z e)))
Without this proposal, this would have to be written
(defmacro frob (&environment e x y &optional (z (locally
(declare
(optimize
(safety 3)
(space 3)
(speed 0)))
(foo x y))))
(locally (declare (optimize (safety 3) (space 3) (speed 0)))
(mumble x y z e)))
Rationale:
Wrapping LOCALLY around a form should not change its semantics except
as specified by the declarations, hence the body of a top-level
LOCALLY should be top-level.
A macro cannot have a top-level body unless it expands into a special
form that has a top-level body; otherwise the macro invocation and
the macro expansion would not have identical semantics as top-level
forms. There is no available special form for LOCALLY to macroexpand
into (CLtL doesn't say, but presumably the intent was to expand into
a LET with an empty binding list).
Current practice:
The Zetalisp equivalent of LOCALLY worked to surround top-level forms,
because it was a macro that expanded into COMPILER-LET (stashing the
declarations in a special variable the compiler would look at). This
is of course the wrong way to do declarations, but it shows that the
idea was that you could wrap declarations around a bunch of top-level
forms.
Symbolics Genera 7.4.0 does not implement the proposal (but it does
not implement DECLARATION-SCOPE:LIMITED-HOISTING either). I did
not survey any other implementations.
Cost to Implementors:
A half dozen lines of code in the compiler and a smaller amount
in the interpreter and any program-analyzing programs.
Cost to Users:
None.
Cost of non-adoption:
See the horrible example above.
Performance impact:
None.
Benefits:
More consistent language.
Esthetics:
Improved.
Discussion:
None.
∂09-Mar-89 1603 CL-Compiler-mailer issue EVAL-WHEN-NON-TOP-LEVEL, version 6
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 9 Mar 89 16:03:26 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA05558; Thu, 9 Mar 89 17:01:21 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA10059; Thu, 9 Mar 89 17:01:19 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903100001.AA10059@defun.utah.edu>
Date: Thu, 9 Mar 89 17:01:17 MST
Subject: issue EVAL-WHEN-NON-TOP-LEVEL, version 6
To: cl-compiler@sail.stanford.edu
-------
∂09-Mar-89 1609 CL-Compiler-mailer issue DEFINING-MACROS-NON-TOP-LEVEL, version 8
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 9 Mar 89 16:09:36 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA05860; Thu, 9 Mar 89 17:07:30 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA10083; Thu, 9 Mar 89 17:07:27 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903100007.AA10083@defun.utah.edu>
Date: Thu, 9 Mar 89 17:07:26 MST
Subject: issue DEFINING-MACROS-NON-TOP-LEVEL, version 8
To: cl-compiler@sail.stanford.edu
The definition of "top-level" has been moved to issue
EVAL-WHEN-NON-TOP-LEVEL. Otherwise this writeup is mostly
unchanged.
Forum: Compiler
Issue: DEFINING-MACROS-NON-TOP-LEVEL
References: CLtL p. 66-70, 143
Issue EVAL-WHEN-NON-TOP-LEVEL
Issue COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS
Issue COMPILER-LET-CONFUSION
Category: CLARIFICATION, ENHANCEMENT
Edit History: 6-May-88, V1 by Sandra Loosemore
9-Jun-88, V2 by Sandra Loosemore
12-Sep-88, V3 by Sandra Loosemore (fix garbled section 4)
21-Sep-88, V4 by Sandra Loosemore (clarify section 5)
16-Dec-88, V5 by Sandra Loosemore (major restructuring)
31-Dec-88, V6 by Sandra Loosemore (wording clarifications)
07-Jan-89, V7 by Sandra Loosemore (add example)
09-Mar-89, V8 by Sandra Loosemore (more restructuring)
Status: Ready for release
Problem Description:
CLtL leaves the interpretation of defining forms such as DEFMACRO and
DEFVAR that appear in other than top-level locations unclear.
On page 66, it is stated: "It is not illegal to use these forms at
other than top level, but whether it is meaningful to do so depends on
context. Compilers, for example, may not recognize these forms
properly in other than top-level contexts". At least one implementation
has interpreted this to mean that it is permissible to simply refuse
to compile defining macros that do not appear at top-level.
Proposal: DEFINING-MACROS-NON-TOP-LEVEL:ALLOW
(1) Remove the language from p. 66 of CLtL quoted above. Clarify that
while defining macros normally appear at top level, it is meaningful
to place them in non-top-level contexts and that the compiler must
handle them properly in all situations. However, the compile-time side
effects described in issue COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS
only take place when the defining macros appear at top-level.
(2) Remove the language on p. 145 of CLtL, which states that macro
functions are always defined in the null lexical environment. Clarify
that all defining macros which create functional objects (including
DEFMACRO, DEFTYPE, DEFINE-SETF-METHOD, and the complex form of
DEFSETF, as well as DEFUN) must ensure that those functions are
defined in the lexical environment in which the defining form is
evaluated.
(3) Specify that top-level forms in a file being compiled are
guaranteed to be processed sequentially. The order in which
non-top-level subforms of a top-level form are processed by the
compiler is explicitly left unspecified.
Rationale:
This proposal makes the rules for when defining macros cause
compile-time side effects to be exactly the same as the rules for when
(EVAL-WHEN (COMPILE) ...) causes compile-time evaluation. This
provides a simple implementation technique.
Item (3) serves two purposes. First, it guarantees users that
compile-time side-effects from top-level EVAL-WHEN forms or defining
macros will happen in the correct order; programmers can depend upon
the compile-time side-effects of a top-level form being visible during
the compilation of subsequent forms. Second, it allows compilers to
perform certain kinds of source-to-source transformations that change
the order of subforms.
For instance, the following example from CLtL
(let ((old-count *access-count*))
(unwind-protect
(progn
(incf *access-count*)
(perform-access))
(setq *access-count* old-count)))
is entirely equivalent to:
(let ((old-count *access-count*))
(let ((thunk #'(lambda () (setq *access-count* old-count))))
(unwind-protect
(progn
(incf *access-count*)
(perform-access))
(funcall thunk))))
(This is a real example from the A-Lisp compiler, which implements
UNWIND-PROTECT by having it push a "thunk" to perform the cleanup
actions onto the catch stack before executing the protected form.)
Current Practice:
Most implementations do allow defining macros in non-top-level places.
However, the rules for when they cause compile-time side-effects are
not always the same as those for EVAL-WHEN. This is the case in
Lucid Common Lisp, for example.
Cost to implementors:
Implementations that currently don't compile defining macros correctly
when they appear at non-top-level will have to be changed.
Cost to users:
None. This is a compatible extension.
Benefits:
The notion of defining macros as being somehow special when they
appear at top-level is removed, since their behavior can be explained
using EVAL-WHEN as a primitive. Allowing defining macros to appear
anywhere instead of restricting them to certain positions results in a
cleaner language design.
Discussion:
This proposal is consistent with the behavior specified in proposal
EVAL-WHEN-NON-TOP-LEVEL:GENERALIZE-EVAL. In particular, if the compile
time side-effects for defining macros specified in proposal
COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS:CLARIFY are implemented using
EVAL-WHEN, the "right" compiler behavior for defining macros at
non-top-level will happen automatically.
-------
∂09-Mar-89 1619 Common-Lisp-Object-System-mailer issue CLOS-MACRO-COMPILATION
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 9 Mar 89 16:19:35 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA06189; Thu, 9 Mar 89 17:17:27 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA10108; Thu, 9 Mar 89 17:17:24 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903100017.AA10108@defun.utah.edu>
Date: Thu, 9 Mar 89 17:17:22 MST
Subject: issue CLOS-MACRO-COMPILATION
To: Gregor.pa@Xerox.COM
Cc: cl-compiler@sail.stanford.edu, common-lisp-object-system@sail.stanford.edu
In-Reply-To: Gregor.pa@Xerox.COM, Thu, 23 Feb 89 20:23 PST
Have you made any progress yet on settling on your new model of how
the CLOS defining macros work? I have opened a new cl-compiler issue,
CLOS-MACRO-COMPILATION, for this, but I don't yet have a proposal.
(Or even a problem statement written down, for that matter.) I am
planning to distribute the rest of our issues to X3J13 early next
week and would like to have at least a draft ready by then. If you
don't think that's possible, I will have to say in our report that
we need an extension to get this issue resolved.
-Sandra
-------
∂09-Mar-89 1606 CL-Compiler-mailer issue EVAL-WHEN-NON-TOP-LEVEL, version 6
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 9 Mar 89 16:06:24 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA05662; Thu, 9 Mar 89 17:04:14 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA10071; Thu, 9 Mar 89 17:04:10 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903100004.AA10071@defun.utah.edu>
Date: Thu, 9 Mar 89 17:04:08 MST
Subject: issue EVAL-WHEN-NON-TOP-LEVEL, version 6
To: cl-compiler@sail.stanford.edu
[How did the mailer lose the text to this message when I sent it out
before?]
With the help of Pitman and Moon, I have cleaned up the presentation
of the GENERALIZE-EVAL proposal from version 5. Among other things, I
have moved the definition of "top-level" here (from issue
DEFINING-MACROS-NON-TOP-LEVEL), so there is now a more coherent
description of how the compiler processes top-level forms. (If
anybody has been looking over the working draft of the standard, that
description is going to end up in section 4.2.)
Issue: EVAL-WHEN-NON-TOP-LEVEL
Forum: Compiler
References: EVAL-WHEN (CLtL pp69-70),
Issue DEFINING-MACROS-NON-TOP-LEVEL
Issue COMPILED-FUNCTION-REQUIREMENTS
Issue IN-PACKAGE-FUNCTIONALITY
Issue LOCALLY-TOP-LEVEL
Category: CLARIFICATION/CHANGE
Edit History: 06-May-88, Version 1 by Sandra Loosemore
16-Dec-88, Version 2 by Loosemore (alternate direction)
30-Dec-88, Version 3 by Loosemore (minor wording changes)
07-Jan-89, Version 4 by Loosemore (update discussion)
09-Feb-89, Version 5 by Pitman and Moon (some major changes)
09-Mar-89, Version 6 by Loosemore (clean up wording)
Status: Ready for release
Problem Description:
The current description of how the compiler should handle EVAL-WHEN
only makes sense when it appears as a top-level form in the file being
compiled. Is it legitimate for EVAL-WHEN to appear in non-top-level
locations? Even if it is legitimate, what does it mean?
Another issue, referred to here as ``the EVAL-WHEN shadowing problem,''
is that some people have complained that shadowing the symbols EVAL,
COMPILE, or LOAD means that you have to also either shadow EVAL-WHEN
and define it to recognize the new symbol, or else you must resign
yourself to writing (EVAL-WHEN (... LISP:EVAL ...) ...),etc. all over.
While the goal here is not to solve this problem, it might be possible
to solve both problems at once.
There are two proposals presented here, GENERALIZE-EVAL and
GENERALIZE-EVAL-NEW-KEYWORDS.
Background/Analysis:
The proposal which follows was constructed with the following goals
in mind:
1. The lexical and dynamic environment for the EVAL-WHEN body should
be the same for each situation. That is, the body should ``mean
the same thing'' regardless of which situation is being processed.
2. The evaluation context for EVAL-WHEN should be the current
lexical environment.
3. At execution time, EVAL-WHEN should always return the result of
its last form if execution of the body occurred, or NIL if the
body was not executed.
4. If a top-level EVAL-WHEN has a LOAD keyword, its body should
inherit top-level-ness during normal processing. This permits the
use of (EVAL-WHEN (EVAL COMPILE LOAD) ...) at top-level to mean
simply "Do whatever would normally be done for this body, but
also do something at compile time." This, in turn, will later be
the key to allowing defining forms to be usefully described in
terms of EVAL-WHEN.
5. Non-top-level expressions should have no effect until they are
executed. This is the key to making sure that any necessary
environment is present. Since the COMPILE keyword forces effects
to occur earlier than execution time, it follows from this that
any correct solution must not allow the COMPILE keyword to have
an effect at other than top-level.
To accomplish these goals, we formulated the following model:
The purpose of EVAL-WHEN is to accomodate the fact that some of the
semantic processing of an expression may usefully be partitioned
between compile time and run time in some circumstances.
(EVAL-WHEN (EVAL) <code>)
describes a general technique for accomplishing some particular goal
at normal program execution time. However, the pair of expressions
(EVAL-WHEN (COMPILE) <code-A>)
(EVAL-WHEN (LOAD) <code-B>)
can be used to describe an alternate technique for implementing part
of the effect (A) at compile-time, and part of the effect (B) at
load-time.
Proposal (EVAL-WHEN-NON-TOP-LEVEL:GENERALIZE-EVAL):
Replace the description of EVAL-WHEN with the following:
EVAL-WHEN ({situation}*) {form}* [Special Form]
The body of an EVAL-WHEN form is processed as an implicit PROGN, but
only in the situations listed. Each SITUATION must be a symbol,
either COMPILE, LOAD, or EVAL.
The use of COMPILE and LOAD controls whether and when processing
occurs for top-level forms. The use of EVAL controls whether
processing occurs for non-top-level forms.
The EVAL-WHEN construct may be more precisely understood in terms of
a model of how the file compiler, COMPILE-FILE, processes forms in a
file to be compiled.
Successive forms are read from the file by the file compiler using
READ. These top-level forms are normally processed in what we call
`not-compile-time' mode. There is one other mode, called
`compile-time-too' mode, which can come into play for top-level
forms. The EVAL-WHEN special form is used to annotate a program
in a way that allows the program doing the processing to select
the appropriate mode.
Processing of top-level forms in the file compiler works as follows:
* If the form is a macro call, it is expanded and the result is
processed as a top-level form in the same processing mode
(compile-time-too or not-compile-time).
* If the form is a PROGN form, each of its body forms is
sequentially processed as top-level forms in the same processing
mode.
* If the form is a COMPILER-LET, MACROLET, or SYMBOL-MACROLET,
the file compiler makes the appropriate bindings and recursively
processes the body forms as an implicit top-level PROGN with those
bindings in effect, in the same processing mode.
* If the form is an EVAL-WHEN form, it is handled according to
the following table:
COMPILE LOAD EVAL compile-time-too Action
Yes Yes -- -- Process body in compile-time-too mode
No Yes Yes Yes Process body in compile-time-too mode
No Yes Yes No Process body in not-compile-time mode
No Yes No -- Process body in not-compile-time mode
Yes No -- -- Evaluate body
No No Yes Yes Evaluate body
No No Yes No do nothing
No No No -- do nothing
"Process body" means to process the body as an implicit top-level
PROGN. "Evaluate body" means to evaluate the body forms as in
implicit PROGN in the dynamic execution context of the compiler and
in the lexical environment in which the EVAL-WHEN appears.
* Otherwise, the form is a top-level form that is not one of the
special cases. If in compile-time-too mode, the compiler first
evaluates the form and then performs normal compiler processing
on it. If in not-compile-time mode, only normal compiler
processing is performed. [The nature of this processing is
defined more precisely in issue COMPILED-FUNCTION-REQUIREMENTS.]
Any subforms are treated as non-top-level forms.
For an EVAL-WHEN form that is not a top-level form in the file compiler
(that is, one of: in the interpreter; in COMPILE; or in the file
compiler but not at top-level), if the EVAL situation is specified,
its body is treated as an implicit PROGN. Otherwise, the EVAL-WHEN
form returns NIL.
Clarifications/Consequences:
The following effects are logical consequences of the above proposal:
* It is never the case that the execution of a single EVAL-WHEN
expression will execute the body code more than once.
* The keyword `EVAL' is a misnomer because execution of
the body need not be done by EVAL. In compiled code, such as
(DEFUN FOO () (EVAL-WHEN (EVAL) (PRINT 'FOO)))
the call to PRINT should be compiled.
* Macros intended for use in top-level forms should arrange for all
side-effects to be done by the forms in the macro expansion.
The macro-expander itself should not do the side-effects.
Wrong: (defmacro foo ()
(really-foo)
`(really-foo))
Right: (defmacro foo ()
`(eval-when (compile eval load) (really-foo)))
Adherence to this convention will mean that such macros will behave
intuitively when placed in non-top-level positions.
* Placing a variable binding around an EVAL-WHEN reliably captures the
binding because the `compile-time-too' mode cannot occur (because
introducing a variable binding would mean we were not at top level).
For example,
(LET ((X 3))
(EVAL-WHEN (EVAL LOAD COMPILE) (PRINT X)))
will print 3 at execution [load] time, and will not print anything at
compile time. This is important so that expansions of DEFUN and
DEFMACRO can be done in terms of EVAL-WHEN and can correctly capture
the lexical environment.
(DEFUN BAR (X) (DEFUN FOO () (+ X 3)))
might expand into
(DEFUN BAR (X)
(PROGN (EVAL-WHEN (COMPILE)
(COMPILER::NOTICE-FUNCTION-DEFINITION 'FOO '(X)))
(EVAL-WHEN (EVAL LOAD)
(SETF (SYMBOL-FUNCTION 'FOO) #'(LAMBDA () (+ X 3))))))
which would be treated the same as
(DEFUN BAR (X)
(SETF (SYMBOL-FUNCTION 'FOO) #'(LAMBDA () (+ X 3))))
by the above rules.
Test Cases:
;; #1: The EVAL-WHEN in this case is not at top-level, so only the EVAL
;; keyword is considered. At compile time, this has no effect.
;; At load time (if the LET is at top level), or at execution time
;; (if the LET is embedded in some other form which does not execute
;; until later) this sets (SYMBOL-FUNCTION 'FOO1) to a function which
;; returns 1.
(LET ((X 1))
(EVAL-WHEN (EVAL LOAD COMPILE)
(SETF (SYMBOL-FUNCTION 'FOO1) #'(LAMBDA () X))))
;; #2: If this expression occurs at the top-level of a file to be compiled,
;; it has BOTH a compile time AND a load-time effect of setting
;; (SYMBOL-FUNCTION 'FOO2) to a function which returns 2.
(EVAL-WHEN (EVAL LOAD COMPILE)
(LET ((X 2))
(EVAL-WHEN (EVAL LOAD COMPILE)
(SETF (SYMBOL-FUNCTION 'FOO2) #'(LAMBDA () X)))))
;; #3: If this expression occurs at the top-level of a file to be compiled,
;; it has BOTH a compile time AND a load-time effect of setting the
;; function cell of FOO3 to a function which returns 3.
(EVAL-WHEN (EVAL LOAD COMPILE)
(SETF (SYMBOL-FUNCTION 'FOO3) #'(LAMBDA () 3)))
;; #4: This always does nothing. It simply returns NIL.
(EVAL-WHEN (COMPILE)
(EVAL-WHEN (COMPILE)
(PRINT 'FOO4)))
;; #5: If this form occurs at top-level of a file to be compiled, FOO5 is
;; printed at compile time. If this form occurs in a non-top-level
;; position, nothing is printed at compile time. Regardless of context,
;; nothing is ever printed at load time or execution time.
(EVAL-WHEN (COMPILE)
(EVAL-WHEN (EVAL)
(PRINT 'FOO5)))
;; #6: If this form occurs at top-level of a file to be compiled, FOO6 is
;; printed at compile time. If this form occurs in a non-top-level
;; position, nothing is printed at compile time. Regardless of context,
;; nothing is ever printed at load time or execution time.
(EVAL-WHEN (EVAL LOAD)
(EVAL-WHEN (COMPILE)
(PRINT 'FOO6)))
Rationale:
This is compatible with any guarantees made by CLtL, and extends the
behavior usefully to non-top-level situations.
This gives a useful meaning to EVAL-WHEN that supports useful and
predictable behavior if defining macros are used in a non-top-level
situation.
Proposal (EVAL-WHEN-NON-TOP-LEVEL:GENERALIZE-EVAL-NEW-KEYWORDS):
As in GENERALIZE-EVAL, but rename the EVAL keyword to :EXECUTE,
the COMPILE keyword to :COMPILE-TOPLEVEL, and LOAD keyword to
:LOAD-TOPLEVEL.
Deprecate the use of keywords EVAL, COMPILE, and LOAD to EVAL-WHEN.
For compatibility, they are supported in EVAL-WHEN
at top-level, but their meaning is not defined elsewhere.
Rationale:
The fact that the situation keywords chosen are not the same as
those now used means that the change can be added in a way that
is truly upward compatible (not only with CLtL but with existing
practice in implementations which have chosen to extend or `clarify'
the definition given in CLtL) since the meaning of EVAL, COMPILE,
and LOAD in non-top-level situations (which was never spelled
out in CLtL) can legitimately differ from the meaning of these
new keywords.
Using other names and/or the keyword package for the names of
situations solves the EVAL-WHEN shadowing problem.
The name `execute' does not promote the confusion that the body of an
EVAL-WHEN must be executed only in the evaluator. It also does not
promote the confusion that the body of an EVAL-WHEN, regardless of when
executed, must run interpreted.
The names `compile-toplevel' and `load-toplevel' emphasize the fact
that these cases are not interesting in non-top-level positions.
Current Practice:
In Symbolics Genera, the interpreter permits EVAL-WHEN in non-top-level
positions in a way that is compatible with this proposal but both the
COMPILE and COMPILE-FILE functions complain about EVAL-WHEN in a
non-top-level position.
Both Lucid Common Lisp and Kyoto Common Lisp already interpret the
EVAL keyword to mean "execute" in non-top-level situations. Both of
these implementations also make (EVAL-WHEN (LOAD) ...) suppress
compile-time "magic" from defining macros such as DEFMACRO.
IIM describes its EVAL-WHEN as:
(defmacro eval-when (situations &body body &environment env)
(if (not (compiler-environment-p env))
(when (member 'eval situations) `(progn ,@body))
(progn
(when (member 'compile situations)
(if (compiler-at-top-level-p env)
(mapc #'eval body)
(warn "Top-level form encountered at non-top-level.")))
(when (member 'load situations) `(progn ,@body)))))
Note that the interpretation of the EVAL situation and the nesting
behavior is different.
Cost to Implementors:
The actual change to EVAL-WHEN in both cases is probably fairly
localized and straightforward to make in most or all implementations.
The second-order costs of proposal GENERALIZE-EVAL will vary depending
on whether existing implementations have extended the definition of
EVAL-WHEN in incompatible ways. If an implementation has made such
extensions, there may be user and system code which depends on them
and the cost of converting that code may be non-trivial. There is
presumably also documentation impact.
Proposal GENERALIZE-EVAL-NEW-KEYWORDS avoids most or all of the
second-order costs of proposal GENERALIZE-EVAL.
The compiler processing for top-level forms might be implemented
something like:
;;; Forms read by the file compiler are passed to PROCESS-TOP-LEVEL-FORM
;;; with a env compile-time-too both NIL.
(defun process-top-level-form (form env compile-time-too)
(setq form (macroexpand form env))
(cond ((not (consp form))
nil)
((eq (car form) 'progn)
(dolist (f (cdr form))
(process-top-level-form f env compile-time-too)))
((eq (car form) 'compiler-let)
(process-compiler-let form env compile-time-too))
((eq (car form) 'macrolet)
(process-macrolet form env compile-time-too))
((eq (car form) 'symbol-macrolet)
(process-symbol-macrolet form env compile-time-too))
((eq (car form) 'eval-when)
(process-eval-when form env compile-time-too))
(t
(if compile-time-too
(internal-eval form env))
(compile-form form env))
))
(defun process-eval-when (form env compile-time-too)
(let* ((situations (cadr form))
(body (cddr form))
(compile-p (member 'compile situations))
(load-p (member 'load situations))
(eval-p (member 'eval situations)))
(cond ((or (and compile-p load-p)
(and eval-p load-p compile-time-too))
(process-top-level-form `(progn ,@body) env t))
(load-p
(process-top-level-form `(progn ,@body) env nil))
((or compile-p
(and eval-p compile-time-too))
(dolist (f body)
(internal-eval f env)))
(t
nil))))
;;; PROCESS-COMPILER-LET, PROCESS-MACROLET, and PROCESS-SYMBOL-MACROLET
;;; do the obvious things.
;;; INTERNAL-EVAL evaluates "form" in lexical environment "env".
Cost to Users:
Technically, none. Either proposal is technically upward compatible
with CLtL.
Proposal GENERALIZE-EVAL might force some extended implementations to
change incompatibly. As such, some users who depend on
implementation-dependent extensions might have to adjust their code
somewhat to deal with those changes.
Proposal GENERALIZE-EVAL-NEW-KEYWORDS does not force implementations
to change incompatibly, so has no forced impact on users.
Cost of Non-Adoption:
EVAL-WHEN is a mess. Using it as the low-level substrate into which
defining macros should expand, and guaranteeing any predictable effects
of those macros in non-top-level situations is currently difficult and
would continue to be so in the absence of some resolution on this issue.
Benefits:
The costs of non-adoption would be avoided: it would be possible to
use EVAL-WHEN in many situations where it cannot currently be used
reliably.
The portability of many existing tools which use EVAL-WHEN internally
in macros will be enhanced.
Aesthetics:
This generalization of the meaning makes the purpose and uses of
EVAL-WHEN less mysterious. In that sense, aesthetics are simplified
somewhat.
Discussion:
The cleanup issue LOCALLY-TOP-LEVEL would make LOCALLY also "pass
through" top-level-ness to its body. The reason why that is not
addressed in this issue is that it involves making LOCALLY a special
form.
Pitman and Moon don't care whether we say `top level,' `top-level,' or
`toplevel.' The spelling choices in this writeup are arbitrary. If
necessary, the proposal GENERALIZE-EVAL-NEW-KEYWORDS could be amended
to propose :COMPILE-TOP-LEVEL, etc.
Pitman, Moon, and Bob Laddaga (a Symbolics Cloe implementor) support
both of these proposals. Pitman and Laddaga have a preference for
GENERALIZE-EVAL-NEW-KEYWORDS. Moon is neutral about which should be
preferred.
Sandra Loosemore says:
I still feel somewhat uncomfortable with the definition of EVAL-WHEN
presented here, mostly because its nesting behavior is so unintuitive
(as in test case number 6). We have also had a hard time in deciding
what the term "top-level" really means; the definition presented here
is rather arbitrary. However, since we have run out of time in which
to come up with acceptable alternatives, I'm willing to go along with
proposal GENERALIZE-EVAL. It is compatible with the description in
CLtL but presented in a more coherent way, and I think it is an
improvement. On the other hand, I don't really like the idea of
changing the names of the keywords; if we are going to make an
incompatible change, the right thing to do would be to throw out
EVAL-WHEN entirely and start from scratch.
-------
∂09-Mar-89 1652 CL-Compiler-mailer cl-compiler issue status as of 3/9
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 9 Mar 89 16:52:46 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA06843; Thu, 9 Mar 89 17:50:41 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA10126; Thu, 9 Mar 89 17:50:39 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903100050.AA10126@defun.utah.edu>
Date: Thu, 9 Mar 89 17:50:38 MST
Subject: cl-compiler issue status as of 3/9
To: cl-compiler@sail.stanford.edu
OK, here's where we stand now.
This is the list of issues which I think will be ready to send out in
time to meet the 2-week rule. The ones marked with an asterisk are
the ones that I've sent out new versions on in the past day or two.
My plan is to mail all of these to X3J13 Monday night, but please
don't wait until the last minute if you have some major complaint that
you haven't told me about yet. As far as I'm aware, none of these
issues require major changes but on some of them I do have some notes
about updating things like current practice or error terminology.
* COMPILE-ENVIRONMENT-CONSISTENCY
COMPILE-FILE-SYMBOL-HANDLING
COMPILED-FUNCTION-REQUIREMENTS
COMPILER-DIAGNOSTICS
* COMPILER-LET-CONFUSION
COMPILER-VERBOSITY
CONSTANT-CIRCULAR-COMPILATION
CONSTANT-COLLAPSING
CONSTANT-COMPILABLE-TYPES
* DEFINING-MACROS-NON-TOP-LEVEL
* EVAL-WHEN-NON-TOP-LEVEL
LOAD-TIME-EVAL
* MACRO-ENVIRONMENT-EXTENT
QUOTE-SEMANTICS
SAFE-CODE
Unless somebody sends me revised proposals Real Soon Now, it doesn't
look like these issues are going to be ready in time to meet the
2-week rule:
CLOS-MACRO-COMPILATION
COMPILE-FILE-ENVIRONMENT
DEFCONSTANT-NOT-WIRED
DEFINE-OPTIMIZER
PROCLAIM-ETC-IN-COMPILE-FILE
SYNTACTIC-ENVIRONMENT-ACCESS (aka MACRO-ENVIRONMENT-CREATOR)
WITH-COMPILATION-UNIT
Issue CLOS-MACRO-COMPILATION is troublesome. If we can't come up with
a proposal before the meeting, I think we'll have to ask for an
extension. I don't think the standard would be complete without
saying something on this issue -- it's not a proposal for an addition
to the language, it's a clarification of the compilation semantics of
a basic language feature.
-Sandra
-------
∂09-Mar-89 1708 CL-Compiler-mailer issue CLOS-MACRO-COMPILATION
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 9 Mar 89 17:08:20 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 554334; Thu 9-Mar-89 20:05:08 EST
Date: Thu, 9 Mar 89 20:04 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: issue CLOS-MACRO-COMPILATION
To: Sandra J Loosemore <sandra%defun@cs.utah.edu>
cc: Gregor.pa@Xerox.COM, cl-compiler@sail.stanford.edu, common-lisp-object-system@sail.stanford.edu
In-Reply-To: <8903100017.AA10108@defun.utah.edu>
Message-ID: <19890310010446.2.MOON@EUPHRATES.SCRC.Symbolics.COM>
Line-fold: No
Date: Thu, 9 Mar 89 17:17:22 MST
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Have you made any progress yet on settling on your new model of how
the CLOS defining macros work?
I didn't get a chance to talk with Gregor when he was here a couple
days ago, but I have made some progress on this myself. However, it's
not ready to show to anyone yet.
I have opened a new cl-compiler issue,
CLOS-MACRO-COMPILATION, for this, but I don't yet have a proposal.
(Or even a problem statement written down, for that matter.) I am
planning to distribute the rest of our issues to X3J13 early next
week and would like to have at least a draft ready by then. If you
don't think that's possible, I will have to say in our report that
we need an extension to get this issue resolved.
What I'm doing won't be ready to show to X3J13 that soon. However,
assuming that eval-when is settled now, I don't think what I'm doing
will have any effect on X3J13 as it should be all at the metaobject
level. I guess I won't know that for sure until I'm done.
∂09-Mar-89 1920 CL-Cleanup-mailer Re: Potential issue: MACRO-SPECIAL-FORMS
Received: from Xerox.COM by SAIL.Stanford.EDU with TCP; 9 Mar 89 19:19:59 PST
Received: from Semillon.ms by ArpaGateway.ms ; 09 MAR 89 19:14:55 PST
Date: Thu, 9 Mar 89 19:14 PST
From: Gregor.pa@Xerox.COM
Subject: Re: Potential issue: MACRO-SPECIAL-FORMS
To: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
cc: Jeff Dalton <jeff%aiai.edinburgh.ac.uk@NSS.Cs.Ucl.AC.UK>, Kent M Pitman
<KMP@STONY-BROOK.SCRC.Symbolics.COM>, cl-compiler@sail.stanford.edu,
cl-cleanup@sail.stanford.edu
Fcc: BD:>Gregor>mail>outgoing-mail-5.text.newest
In-Reply-To: <19890308022127.7.MOON@EUPHRATES.SCRC.Symbolics.COM>
Message-ID: <19890310031441.0.GREGOR@SPIFF.parc.xerox.com>
Line-fold: no
I agree with Moon's position on this.
I have written my own code walker for use in PCL. I keep having to add
special forms to it anyways, and it turns out it is much easier to do
that than deal with screwy "expansions" of "macros".
-------
∂09-Mar-89 2220 CL-Compiler-mailer issue MACRO-ENVIRONMENT-EXTENT, version 2
Received: from ALDERAAN.SCRC.Symbolics.COM ([128.81.41.109]) by SAIL.Stanford.EDU with TCP; 9 Mar 89 22:20:53 PST
Received: from GANG-GANG.SCRC.Symbolics.COM by ALDERAAN.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 280104; Fri 10-Mar-89 01:17:50 EST
Date: Fri, 10 Mar 89 01:18 EST
From: Glenn S. Burke <gsb@ALDERAAN.SCRC.Symbolics.COM>
Subject: issue MACRO-ENVIRONMENT-EXTENT, version 2
To: sandra%defun@cs.utah.edu, cl-compiler@sail.stanford.edu
In-Reply-To: <8903092218.AA09917@defun.utah.edu>
Message-ID: <19890310061817.9.GSB@GANG-GANG.SCRC.Symbolics.COM>
I believe in dynamic extent for macro environments. The efficiency
penalty of doing otherwise increases as more functionality becomes
attached to macro environments.
I have serious doubts that adding a copier for macro environments would
help, because of the quantity of data which might need to be copied:
this could conceivably be a compiler's entire model of a compilation
destination (type proclamations, defmacro definitions, etc).
Most if not all of the examples i've seen which purport to require
indefinite extent for macroexpansion environments would be better done
(easier to read and understand, easier to implement efficiently) by
special constructs designed for the purpose.
∂09-Mar-89 2315 CL-Compiler-mailer Issue SYNTACTIC-ENVIRONMENT-ACCESS
Received: from ECLA.USC.EDU by SAIL.Stanford.EDU with TCP; 9 Mar 89 23:14:51 PST
Date: Thu 9 Mar 89 23:12:17-PST
From: Kim A. Barrett <IIM@ECLA.USC.EDU>
Subject: Issue SYNTACTIC-ENVIRONMENT-ACCESS
To: cl-compiler@SAIL.STANFORD.EDU
cc: iim@ECLA.USC.EDU
Message-ID: <12476822375.25.IIM@ECLA.USC.EDU>
Major changes from version 2.
1. ENVIRONMENT-TARGET => ENVIRONMENT-REMOTE-P
2. Dropped the ENVIRONMENT- prefix from the accessors, and made the env
arguments optional (Gray's suggestion).
3. TYPE and FTYPE information made optional.
4. AUGMENT-ENVIRONMENT: Replaced :lexical argument with :variable argument,
taking on the responsibility of seperating lexicals from specials based on
the :special argument and global proclamations.
5. New stuff: WITH-REMOTE-ENVIRONMENT, RECORD-ENVIRONMENT-CLEANUP, and
MAKE-NULL-ENVIRONMENT.
6. Significant rewrites of various sections in response to comments. Hopefully
this version will be found less confusing and better motivated.
kab
-----
Issue: SYNTACTIC-ENVIRONMENT-ACCESS
References: CLtL Chapter 8: Macros,
Issue MACRO-FUNCTION-ENVIRONMENT,
Issue GET-SETF-METHOD-ENVIRONMENT,
Issue COMPILE-FILE-ENVIRONMENT
Related Issues: Issue FUNCTION-NAME,
Issue PROCLAIM-LEXICAL
Category: ADDITION
Edit history: Version 1, 2-Oct-88, Eric Benson
Version 2, 17-Feb-89, Kim A. Barrett
Version 3, 9-Mar-89, Kim A. Barrett (respond to comments)
Status: For internal discussion
Problem description:
When macro forms are expanded, the expansion function is called with two
arguments: the form to be expanded, and the environment in which the form was
found. The environment argument is of limited utility. The only use
sanctioned currently is as an argument to MACROEXPAND or MACROEXPAND-1 or
passed directly as an argument to another macro expansion function. Recent
cleanup issues propose to allow it as an argument to MACRO-FUNCTION and to
GET-SETF-METHOD.
Implementing the FIND-CLASS and ENSURE-GENERIC-FUNCTION functions of CLOS
requires the ability to distinguish between environments used for compiling to
a file from those used for processing in-core, such as by EVAL or COMPILE.
Resolution of the LOAD-TIME-EVAL issue may also require this information.
This problem has been addressed by the recent cleanup issue
COMPILE-FILE-ENVIRONMENT. Also, it has proven impossible to write a portable
code walker in Common Lisp, due to insufficient access to the information
contained in environments and the inability to augment environments with local
function definitions.
Proposal (SYNACTIC-ENVIRONMENT-ACCESS:ADD-FUNCTIONAL-INTERFACE):
The following functions provide information about syntactic environment
objects. In all of these functions the argument named ENV is an environment
of the sort received by the &ENVIRONMENT argument to a macro or as the
environment argument for EVALHOOK. NIL is the local null lexical environment,
and is therefore considered to be a syntactic environment. All of these
functions should signal an error of type TYPE-ERROR if the value of an
environment argument is not a syntactic environment.
ENVIRONMENT-REMOTE-P env [Function]
Returns true if ENV is a remote environment, false otherwise.
VARIABLE-KIND variable &optional env [Function]
VARIABLE is a symbol. This function returns one of the folloing symbols,
depending on the type of definition or binding which is apparent in ENV.
NIL There is no apparent definition or binding for variable.
:SPECIAL VARIABLE refers to a special variable, either declared or
proclaimed.
:LEXICAL VARIABLE refers to a lexical variable.
:SYMBOL-MACRO VARIABLE refers to a SYMBOL-MACROLET binding.
:CONSTANT VARIABLE refers to a named constant, defined by DEFCONSTANT.
[Note: If issue PROCLAIM-LEXICAL passes, then the :LEXICAL result will also
refer to variables proclaimed lexical.]
Example:
(DEFMACRO KIND-OF-VARIABLE (VAR &ENVIRONMENT ENV)
`',(VARIABLE-KIND VAR ENV))
(DEFVAR A)
(DEFUN TEST ()
(LET (B)
(LET (C)
(DECLARE (SPECIAL C))
(SYMBOL-MACROLET ((D ANYTHING))
(LIST (KIND-OF-VARIABLE A)
(KIND-OF-VARIABLE B)
(KIND-OF-VARIABLE C)
(KIND-OF-VARIABLE D)
(KIND-OF-VARIABLE E))))))
(TEST) -> (:SPECIAL :LEXICAL :SPECIAL :SYMBOL-MACRO NIL)
FUNCTION-KIND function &optional env [Function]
FUNCTION is a function name. This function returns two values, depending on
the type of function definition or function binding which is apparent for
FUNCTION in ENV.
NIL There is no apparent definition for FUNCTION.
:FUNCTION FUNCTION refers to a function.
:MACRO FUNCTION refers to a macro.
:SPECIAL-FORM FUNCTION refers to a special-form.
The second value specifies whether the definition is local or global. If
local, the second value is true, and it is false when the definition is
global.
Some function names may refer to both a global macro and a global special
form. In such a case, the macro takes precedence, and :MACRO is returned as
the first value.
[Note: The use of "function name" rather than "symbol" as the description of
the function argument is intended to be compatible with the various proposals
to extend the syntax of function specifiers. If no such change actually
occurs then this would only refer to symbols.]
Example:
(DEFMACRO KIND-OF-FUNCTION (FUNCTION-NAME &ENVIRONMENT ENV)
`',(FUNCTION-KIND FUNCTION-NAME ENV))
(DEFUN A ())
(DEFMACRO B ())
(DEFUN TEST ()
(FLET ((C ()))
(MACROLET ((D ()))
(MULTIPLE-VALUE-CALL #'LIST
(KIND-OF-FUNCTION A)
(KIND-OF-FUNCTION B)
(KIND-OF-FUNCTION QUOTE)
(KIND-OF-FUNCTION C)
(KIND-OF-FUNCTION D)
(KIND-OF-FUNCTION E)))))
(TEST) -> (:FUNCTION NIL
:MACRO NIL
:SPECIAL-FORM NIL
:FUNCTION T
:MACRO T
NIL NIL)
VARIABLE-TYPE variable &optional env [Function]
VARIABLE is a symbol. This function returns the type specifier
associated with the variable named by the symbol in the environment.
If no explicit association exists, either by PROCLAIM or DECLARE, then
the result is the type specifier T.
The result of this function may not include all the apparent TYPE
declarations for VARIABLE. In particular, an implementation is free to
ignore TYPE declarations, only returning TYPE information which was added to
ENV by a call to AUGMENT-ENVIRONMENT.
Example:
This example assumes that the implementation records type information in the
environment, rather than ignoring it.
(DEFMACRO VARTYPE (VAR &ENVIRONMENT ENV)
`',(VARIABLE-TYPE VAR ENV))
(DEFVAR A 1)
(PROCLAIM '(FIXNUM A))
(DEFUN TEST ()
(LET ((B (AREF "X" 0))
(C 3))
(DECLARE (STRING-CHAR B))
(LIST (VARTYPE A) (VARTYPE B) (VARTYPE C))))
(TEST) -> (FIXNUM STRING-CHAR NIL)
FUNCTION-FTYPE function &optional env [Function]
FUNCTION is a function name. This function returns the functional type
specifier associated with the function in the environment, or the symbol
FUNCTION if there is no functional type declaration or proclamation
associated with the function.
The result of this function may not include all the apparent FTYPE
declarations for FUNCTION. In particular, an implementation is free to
ignore FTYPE declarations, only returning FTYPE information which was added
to ENV by a call to AUGMENT-ENVIRONMENT.
Example:
This example assumes that the implementation records ftype information in the
environment, rather than ignoring it.
(DEFMACRO FUNTYPE (FUN &ENVIRONMENT ENV)
`',(FUNCTION-FTYPE FUN ENV))
(DEFUN A-FUNCTION (X)
(+ X 3))
(PROCLAIM '(FTYPE (FUNCTION (FIXNUM) FIXNUM) A-FUNCTION))
(DEFUN TEST ()
(FLET ((ANOTHER-FUNCTION (X)
(+ X 2)))
(DECLARE (FTYPE (FUNCTION (INTEGER) INTEGER) ANOTHER-FUNCTION))
(LIST (FUNTYPE A-FUNCTION) (FUNTYPE ANOTHER-FUNCTION))))
(TEST) -> ((FUNCTION (FIXNUM) FIXNUM) (FUNCTION (INTEGER) INTEGER))
AUGMENT-ENVIRONMENT env &KEY variable
special
symbol-macro
function
macro
type
ftype [Function]
This function returns a copy of ENV, augmented with the information provided
by the keyword arguments. The arguments are supplied as follows:
:VARIABLE A list of symbols which shall be visible as bound variables in
the new environment. Whether each binding is to be interpreted
as dynamic or lexical depends on the value of the :SPECIAL
argument and SPECIAL proclamations recorded in the environment.
:SPECIAL A list of symbols which shall be visible as dynamic variables
in the new environment. If a symbol in this list also appears
in the :VARIABLE list, then the binding is to be interpreted as
being dynamic.
:SYMBOL-MACRO An alist of symbols and macroexpansions. The new environment
will have local symbol-macro bindings of each symbol to the
corresponding expansion, so that MACROEXPAND will be able to
expand them properly.
:FUNCTION A list of function names which shall be visible as local
function bindings in the new environment.
:MACRO An alist of function names and macroexpansion functions. The
new environment will have local macro bindings of each name to
the corresponding expander function, which will be returned by
MACRO-FUNCTION and used by MACROEXPAND.
:TYPE An alist of symbols and type specifiers. The new environment
will have the type specifier associated with the currently
visible binding of the symbol (defaulting to the global special
binding if there is no binding present). Note that in a single
call to AUGMENT-ENVIRONMENT the :TYPE argument can be thought
of as being processed after all the variable arguments have
been processed, thus allowing bindings and declarations to be
updated with a single call.
:FTYPE An alist of function names and functional type specifiers.
This is analogous to :TYPE, exept operating in the function
namespace.
An error is signaled if any of the symbols specified as keys in the
:SYMBOL-MACRO alist are also included in either the :VARIABLE or :SPECIAL
lists, or are among the key symbols in the :TYPE alist. An error is signaled
if any of the names specified as keys in the :MACRO alist are also included
in the :FUNCTION list. It is an error to modify the list structure of any of
the arguments to this function.
If the ENV argument is a remote environment, then the new environment
returned by this function will also be remote, and the two will refer to the
same model of the remote environment. The extent of the returned environment
is the same as the extent of the argument.
While an environment argument from EVALHOOK may be used as the environment
argument for this function, the reverse is not true. It is an error to
attempt to use the result of AUGMENT-ENVIRONMENT as the environment argument
for EVALHOOK. The environment returned by AUGMENT-ENVIRONMENT may only be
used for syntactic analysis, ie. the functions specified by this proposal and
functions such as MACROEXPAND.
[Note: Some modification to this will be required to support Issue
PROCLAIM-LEXICAL if it is adopted. The changes would involve adding a
:LEXICAL keyword similar to :SPECIAL, and fixing up the error signaling
cases.]
ENVIRONMENT-PROPERTY env name property &optional default
This function and its SETF method allow the association of arbitrary 'global'
properties with names within an environment. An environment can be thought
of as having a local property list associated with any name, and this
function provides access to that property list.
A remote environment may be thought of as an extension of the local
environment. Thus, when this function is applied to a remote environment and
the property is not found, the local environment is then searched.
The association between names and property lists uses EQUAL to match names.
The search of the property list uses EQ to match properties. If the property
is not found, DEFAULT is returned.
Using SETF of ENVIRONMENT-PROPERTY affects all environments which refer to
the same environment model. In particular, if ENV is a local environment
then all local environments are affected, while if ENV is a remote
environment, then all environments refering to the same remote environment
model as the argument are affected.
[Note: The local property list of a name is not necessarily the symbol-plist
of the name, though that is a possible implementation for names which are
symbols.
Note: The use of EQUAL as the matching function for names is to allow for
proposed extensions to function names. If no such extension occurs, then EQ
could be used instead.]
WITH-REMOTE-ENVIRONMENT var &body body [Macro]
Evaluates the BODY forms with VAR bound to a newly created remote
environment. The extent of the environment is the dynamic extent of the
WITH-REMOTE-ENVIRONMENT form. Before the extent is exited, each of the
cleanup functions specified by RECORD-ENVIRONMENT-CLEANUP is called with the
environment as its argument. The order in which the cleanup functions are
called is unspecified.
This is the only specified mechanism by which a new remote environment may be
created.
RECORD-ENVIRONMENT-CLEANUP env function [Function]
Record a function to be called when the extent of the environment is exited.
An error of type TYPE-ERROR is signaled if ENV is not a remote environment.
When the extent of ENV ends, FUNCTION will be called with one argument, an
environment which refers to the same remote environment model as the argument
environment.
[Note: The present author does not like the name of this function.
Suggestions solicited.]
MAKE-NULL-ENVIRONMENT &optional env [Function]
Create and return a NULL lexical environment. The ENV argument is used to
determine whether the new environment should be local or remote. If remote,
the two will refer to the same model of the remote environment. The extent
of the returned environment is the same as the extent of the argument.
Rationale:
This proposal provides a portable interface to environment information which
must otherwise be obtained by implementation-dependent means. The
ENSURE-GENERIC-FUNCTION and FIND-CLASS functions of CLOS require the
ENVIRONMENT-REMOTE-P function and some mechanism similar to that supplied by
the ENVIRONMENT-PROPERTY function. A useful code walker requires the
capability of adding local function definitions to an environment.
Making TYPE and FTYPE information optional continues to allow implementations
the freedom to simply ignore all such declarations.
WITH-REMOTE-ENVIRONMENT and RECORD-ENVIRONMENT-CLEANUP provide functionality
which some people believe is needed to implement some of the remote metaobject
mechanisms in CLOS.
MAKE-NULL-ENVIRONMENT is provided for use by code-walkers, so that they can
handle occurances of special-forms like LOAD-TIME-EVAL, which requires a NULL
lexical environment.
Cost to Implementors:
Most implementations already record some of this information in some form.
Providing these functions should not be too difficult, but it is a more than
trivial amount of work.
Cost to Users:
This change is upward compatible with user code.
Current practice:
No implementation provides all of this interface currently. Portable Common
Loops defines a subset of this functionality for its code walker and
implements it on a number of diffent versions of Common Lisp. IIM uses the
functionality provided by ENVIRONMENT-REMOTE-P and ENVIRONMENT-PROPERTY (under
other names) to implement the association between names and remote metaobjects
(macro and type definitions, CLOS remote metaobjects, &etc).
Discussion:
The first version of this proposal expressly did not deal with the objects
which are used as environments by EVALHOOK. This version is extended to
support them in the belief that such environments share a lot of functionality
with the syntactic environments needed by a compiler. While the two types of
environments may have very different implementations, there are many
operations which are reasonable to perform on either type, including all of
the accessor functions described by this proposal.
AUGMENT-ENVIRONMENT currently requires signaling an error when symbol-macro
names match variable names in the same call. This could be reduced to "should
signal". By requiring the error signaling, this proposal is compatable with
Proposal SYMBOL-MACROLET-DECLARE:ALLOW, which says
"... signals an error if a SPECIAL declaration names one of the symbols
being defined as a symbol-macrolet."
Maintaining compatability with the SYMBOL-MACROLET-DECLARE proposal allows
fairly trivial implementations of the SYMBOL-MACROLET special-form in terms of
the AUGMENT-ENVIRONMENT function.
An possible alternative syntax for WITH-REMOTE-ENVIRONMENT might be
WITH-REMOTE-ENVIRONMENT (var &key) &body body
Can anyone suggest candidates for keyword options? We could do this even if
we can't think of any immediately, leaving room for implementation-specific
extensions. One candidate option that some implementations might want would
be to specify a target machine for the compilation.
Kim Barrett originally suggested that WITH-COMPILATION-UNIT should provide the
mechanism for creating new remote environments. However, it has been
suggested that WITH-COMPILATION-UNIT is intended to serve a somewhat different
purpose. Also, that proposal doesn't seem to be getting much attention due to
time constraints, and this proposal needs to have such a mechanism specified.
≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠≠
-------
∂10-Mar-89 0626 CL-Compiler-mailer Re: Issue: LOCALLY-TOP-LEVEL (Version 1)
Received: from RELAY.CS.NET by SAIL.Stanford.EDU with TCP; 10 Mar 89 06:21:11 PST
Received: from relay2.cs.net by RELAY.CS.NET id aa02326; 10 Mar 89 9:10 EST
Received: from draper.com by RELAY.CS.NET id aa24106; 10 Mar 89 9:03 EST
Date: Fri, 10 Mar 89 08:08 EST
From: "Steve Bacher (Batchman)" <SEB1525@draper.com>
Subject: Re: Issue: LOCALLY-TOP-LEVEL (Version 1)
To: cl-compiler@SAIL.STANFORD.EDU
X-VMS-To: CL-COMPILER,SEB1525
Couldn't LOCALLY be a macro that expands into a MACROLET with a null binding
list (assuming that the body of a MACROLET retains top-level-ness)?
∂10-Mar-89 0836 CL-Compiler-mailer Re: Issue: LOCALLY-TOP-LEVEL (Version 1)
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 10 Mar 89 08:36:04 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA23275; Fri, 10 Mar 89 09:33:58 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA10605; Fri, 10 Mar 89 09:33:50 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903101633.AA10605@defun.utah.edu>
Date: Fri, 10 Mar 89 09:33:48 MST
Subject: Re: Issue: LOCALLY-TOP-LEVEL (Version 1)
To: "Steve Bacher (Batchman)" <SEB1525@draper.com>
Cc: cl-compiler@SAIL.STANFORD.EDU
In-Reply-To: "Steve Bacher (Batchman)" <SEB1525@draper.com>, Fri, 10 Mar 89 08:08 EST
> Couldn't LOCALLY be a macro that expands into a MACROLET with a null binding
> list (assuming that the body of a MACROLET retains top-level-ness)?
I suppose so, but that would still require a change to so that the standard
would *require* it to expand into a MACROLET (and not a LET or something
else).
-Sandra
-------
∂10-Mar-89 0834 CL-Compiler-mailer Re: Issue SYNTACTIC-ENVIRONMENT-ACCESS
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 10 Mar 89 08:34:30 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA23239; Fri, 10 Mar 89 09:32:24 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA10593; Fri, 10 Mar 89 09:32:18 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903101632.AA10593@defun.utah.edu>
Date: Fri, 10 Mar 89 09:32:16 MST
Subject: Re: Issue SYNTACTIC-ENVIRONMENT-ACCESS
To: Kim A. Barrett <IIM@ECLA.USC.EDU>
Cc: cl-compiler@SAIL.STANFORD.EDU
In-Reply-To: Kim A. Barrett <IIM@ECLA.USC.EDU>, Thu 9 Mar 89 23:12:17-PST
Well, this writeup looks better than the last one, but I still don't
think it's quite ready to be voted on yet.
I have some editorial comments on the last paragraph of the problem
description:
> Implementing the FIND-CLASS and ENSURE-GENERIC-FUNCTION functions of CLOS
> requires the ability to distinguish between environments used for compiling to
> a file from those used for processing in-core, such as by EVAL or COMPILE.
> Resolution of the LOAD-TIME-EVAL issue may also require this information.
> This problem has been addressed by the recent cleanup issue
> COMPILE-FILE-ENVIRONMENT. Also, it has proven impossible to write a portable
> code walker in Common Lisp, due to insufficient access to the information
> contained in environments and the inability to augment environments with local
> function definitions.
I'm not sure that saying "CLOS requires" this is entirely accurate.
My impression is that it would be possible to implement the
functionality described in Chapters 1 & 2 without this. It mentions
that an environment object might be used to distinguish between
compile-time and run-time environments, but doesn't specify any
different behavior for the two cases. Perhaps more accurate wording
would be to say "the CLOS meta-object protocol assumes...".
I don't see the connection to LOAD-TIME-EVAL. There's certainly
nothing in the current writeup that depends on users being able to look
at the contents of environment objects.
My understanding was that this issue subsumed issue COMPILE-FILE-ENVIRONMENT.
I'd agree that writing a truly portable code-walker is impossible, but
not for the reason you cite. (The real problem is macros that expand
into nonportable syntax.) How about saying that not having environment
accessors makes it "difficult".
> AUGMENT-ENVIRONMENT env &KEY variable
> special
> symbol-macro
> function
> macro
> type
> ftype [Function]
>
I have a couple of suggestions here.
How about combining the SPECIAL, TYPE, and FTYPE arguments into a
DECLARE argument that takes a list of "decl-spec"s? Breaking them up
requires you to do your own parsing of declarations. This would also
solve the problem of what to do if a LEXICAL declaration is added to
the language.
Also, I'd suggest that the values for the :SYMBOL-MACRO and :MACRO
arguments be supplied in the form of the list that is the CADR of the
SYMBOL-MACROLET or MACROLET form. Maybe this is what you intended,
but it's not clear. Anyway, the rationale for this is that if you
want the :MACRO argument to be real "macro functions", people who
write code walkers would have to write their own macro argument
destructuring code, which is really a pain. Have the implementation
do it.
> ENVIRONMENT-PROPERTY env name property &optional default
I'm still not convinced that we really need this feature. It seems
more like a mechanism that implementations might use internally. I'd
suggest having two proposals: one that includes everything but this,
and one that adds this feature. I don't think "this is needed to
implement CLOS" is a good enough excuse for adding something to the
language. There are lots of things necessary to implement other parts
of the language that we haven't made visible to the user.
> WITH-REMOTE-ENVIRONMENT var &body body [Macro]
>
> RECORD-ENVIRONMENT-CLEANUP env function [Function]
Yuck. Why not use a syntax like unwind-protect?
WITH-REMOTE-ENVIRONMENT var protected-form {cleanup-form}*
> MAKE-NULL-ENVIRONMENT &optional env [Function]
I don't think this function is necessary. When your code walker
encounters a LOAD-TIME-VALUE form, you presumably still have a handle
on the null, remote environment that originally got created with
WITH-REMOTE-ENVIRONMENT. All this function does is retrieve it,
right?
-Sandra
-------
∂10-Mar-89 0916 CL-Cleanup-mailer issue IN-PACKAGE-FUNCTIONALITY, version 7
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 10 Mar 89 09:15:59 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA24750; Fri, 10 Mar 89 10:13:52 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA10642; Fri, 10 Mar 89 10:13:50 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903101713.AA10642@defun.utah.edu>
Date: Fri, 10 Mar 89 10:13:48 MST
Subject: issue IN-PACKAGE-FUNCTIONALITY, version 7
To: cl-cleanup@sail.stanford.edu
Cc: cl-compiler@sail.stanford.edu
Here's a new version of the writeup for this issue. Since Pitman has
indicated he wants to withdraw the SELECT-ONLY proposal, it has gone
away. I also fixed the typo noted by Moon.
Issue: IN-PACKAGE-FUNCTIONALITY
References: IN-PACKAGE (p182-183)
Category: CHANGE
Edit history: 07-Jul-88, Version 1 by Pitman
7-Oct-88, Version 2 by Masinter (discussion)
8-Dec-88, Version 3 by Masinter
12-Dec-88, Version 4 by Masinter
20-Jan-89, Version 5 by Loosemore
30-Jan-89, Version 6 by Loosemore
10-Mar-89, Version 7 by Loosemore
Related-Issues: DEFPACKAGE (passed)
COMPILE-FILE-SYMBOL-HANDLING
Problem Description:
There are two typical uses for IN-PACKAGE -- to define/create a package
and to select a package. The fact that these two purposes have been
given to the same function has led to reduced error checking.
A more general problem is that the "Put In Seven Extremely Randoms"
convention described in CLtL is now recognized by many people as being
unsatisfactory for both package definition and package selection.
The DEFPACKAGE macro provides a much cleaner mechanism for package
definition, but there is still a need for a convenient way to select
a package that has well-defined compilation semantics.
Proposal (IN-PACKAGE-FUNCTIONALITY:NEW-MACRO):
Add a new macro:
SELECT-PACKAGE name [macro]
This macro causes *PACKAGE* to be set to the package named NAME,
which must be a symbol or string. An error is signalled if the
package does not already exist. Everything this macro does is also
performed at compile time if the call appears at top-level.
Remove the function IN-PACKAGE from the standard.
Remove the second paragraph of section 11.7 in CLtL. (This includes
the requirement for special compile-time treatment of the various
package functions.)
Rationale:
This could allow improved error checking and modularity, with only
minimal loss of functionality.
The rationale for proposing SELECT-PACKAGE as a replacement for
IN-PACKAGE, rather than simply changing IN-PACKAGE to have this
behavior, is that such an incompatible change would be confusing to
many people, and would make it more difficult to detect obsolete
usages. There is nothing in this proposal that would prevent
implementations from continuing to provide IN-PACKAGE as an extension.
Making SELECT-PACKAGE a macro rather than a function means that there
is no need to require COMPILE-FILE to handle it specially. Since
DEFPACKAGE is also defined to side-effect the compilation environment,
there is no need to require any of the package functions to be treated
specially by the compiler.
The language in section 11.7 of CLtL puts the burden on
implementations of ensuring that all symbols in a file which is
compiled and loaded end up in the same package that they would if the
source file were loaded interpretively. No implementation can
possibly meet this requirement because, in general, the runtime
behavior of the program cannot be predicted by the compiler.
Current Practice:
Probably no one implements this behavior exactly since it's an
incompatible change to CLtL.
Cost to Implementors:
The SELECT-PACKAGE macro can be implemented trivially by using
EVAL-WHEN in its expansion:
(defmacro select-package (name)
`(eval-when (eval compile load)
(setq *package*
(or (find-package ',name)
(error "Package ~s does not exist." ',name)))))
The changes required to COMPILE-FILE to remove the magic treatment
of the package functions are also likely to be small.
Cost to Users:
In most cases, minor syntactic changes to some files would be
necessary. Programmers that are now using the "Put In Seven
Extremely Randoms" convention will probably find it straightforward
to convert their code to do a DEFPACKAGE followed by a
SELECT-PACKAGE.
Cost of Non-Adoption:
The specification of COMPILE-FILE will be much more difficult to
understand.
The standard will require compilers to solve the halting problem.
Benefits:
Modular package declarations would be encouraged and errors due
to demand-creation of packages would be easier to detect.
The specification of COMPILE-FILE will be simplified.
There will be a clear statement of the requirements for program
conformance, as relating to usage of packages.
Aesthetics:
The fact that IN-PACKAGE is currently ambiguous about intent (whether
the package should exist already or not) is clearly not aesthetic.
Removing it can't be any worse.
The fact that the currently stated requirements for handling of
the package functions by the compiler are not implementable is
clearly not aesthetic.
Discussion:
The dual use of IN-PACKAGE has not been helpful and is confusing.
Some people may find proposal NEW-MACRO more palatable if it merely
deprecated IN-PACKAGE, instead of removing it entirely.
Loosemore and Moon support proposal IN-PACKAGE-FUNCTIONALITY:NEW-MACRO.
Pitman says:
I support NEW-MACRO, though would really prefer you change "remove" to
"deprecate". Making this an incompatible change is gratuitous.
-------
∂10-Mar-89 1000 CL-Cleanup-mailer issue IN-PACKAGE-FUNCTIONALITY, version 7
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 10 Mar 89 09:59:57 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via INTERNET with SMTP id 554628; 10 Mar 89 12:55:40 EST
Date: Fri, 10 Mar 89 12:55 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: issue IN-PACKAGE-FUNCTIONALITY, version 7
To: Sandra J Loosemore <sandra%defun@cs.utah.edu>
cc: cl-cleanup@sail.stanford.edu, cl-compiler@sail.stanford.edu
In-Reply-To: <8903101713.AA10642@defun.utah.edu>
Message-ID: <19890310175523.4.MOON@EUPHRATES.SCRC.Symbolics.COM>
IN-PACKAGE-FUNCTIONALITY:NEW-MACRO is okay with me.
∂10-Mar-89 1258 Common-Lisp-Object-System-mailer issue CLOS-MACRO-COMPILATION, version 1
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 10 Mar 89 12:58:24 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA03202; Fri, 10 Mar 89 13:56:09 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA10809; Fri, 10 Mar 89 13:56:06 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903102056.AA10809@defun.utah.edu>
Date: Fri, 10 Mar 89 13:56:05 MST
Subject: issue CLOS-MACRO-COMPILATION, version 1
To: cl-compiler@sail.stanford.edu, common-lisp-object-system@sail.stanford.edu
In-Reply-To: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>, Thu, 9 Mar 89 20:04 EST
Here is a first cut at getting something written down on this issue.
This is mostly extracted from David Gray's list of questions and
Moon's responses. To me it seems like the question where there is the
most disagreement is whether or not classes are required to be
instantiable at compile-time, so I have made two proposals. MINIMAL
leaves this unspecified, and NOT-SO-MINIMAL requires them to be
instantiable.
Forum: Compiler
Issue: CLOS-MACRO-COMPILATION
References: CLOS chapters 1 & 2 (88-002R)
CLOS chapter 3 (89-003)
Issue COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS
Issue DEFINING-MACROS-NON-TOP-LEVEL
Category: CLARIFICATION
Edit History: V1, 10 Mar 1989, Sandra Loosemore
Status: **DRAFT**
Problem Description:
Do the CLOS defining macros (DEFCLASS, DEFMETHOD, DEFGENERIC, and
DEFINE-METHOD-COMBINATION) have compile-time side-effects similar
to those for DEFSTRUCT or DEFMACRO?
A part of the problem is that we do not currently have a full
understanding of all the issues involved. In particular, work on
defining the CLOS meta-object protocol is still in progress. The goal
is to say enough about the behavior of these macros in the standard so
that users can use them portably in compiled code, but to leave as
much of the behavior as possible unspecified to avoid placing undue
restrictions on the meta-object protocol.
There are two proposals, MINIMAL and NOT-SO-MINIMAL.
Proposal CLOS-MACRO-COMPILATION:MINIMAL:
State that top-level calls to the CLOS defining macros have the
following compile-time side-effects. Any other compile-time behavior
is explicitly left unspecified.
DEFCLASS:
* The class name becomes a type specifier which may appear in
subsequent type declarations.
* The class name can be used to name a superclass in a subsequent
DEFCLASS.
* The class name can be used as a specializer in a subsequent
DEFMETHOD.
DEFGENERIC:
* The generic function can be referenced in a subsequent DEFMETHOD.
* The generic function is not callable at compile-time.
DEFMETHOD:
* The method is not callable at compile-time. If there is a generic
function with the same name at compile-time, compiling a DEFMETHOD
will not add the method to that generic function.
[This also seems to imply that tests for existence of the generic
function, lambda-list congruence, etc. must not happen until
load time.]
DEFINE-METHOD-COMBINATION:
* The method combination can be used in a subsequent DEFGENERIC. If it
is referenced, the body of a long form of method combination must be
evaluable at compile-time.
Rationale:
The compile-time behavior of DEFCLASS is similar to DEFSTRUCT or
DEFTYPE. DEFGENERIC and DEFMETHOD are similar to DEFUN.
DEFINE-METHOD-COMBINATION is similar to DEFMACRO or DEFSETF.
Proposal CLOS-MACRO-COMPILATION:NOT-SO-MINIMAL:
This is the same as proposal MINIMAL, except under DEFCLASS add:
* The class may be instantiated at compile-time. Provided the
appropriate methods are also defined at compile-time, this implies:
- The class can be used as the :METACLASS option of a later DEFCLASS.
- It can be used as the :GENERIC-FUNCTION-CLASS or :METHOD-CLASS option
of a DEFGENERIC, GENERIC-FUNCTION, GENERIC-FLET, or GENERIC-LABELS.
Rationale:
Being able to instantiate a class at compile-time is somewhat more
convenient for users.
Current Practice:
The items listed under DEFCLASS in proposal MINIMAL are fairly standard
programming style.
Flavors does not support compile-time instantiation of classes. It
does not make method combinations available at compile-time either, but
Moon considers that to be a bad design choice.
Cost to implementors:
Unknown.
Cost to users:
Unknown, but probably fairly small.
Note that for proposal NOT-SO-MINIMAL, users still have to ensure that
any methods on the class which may be invoked at compile-time are
fully defined. This includes the INITIALIZE-INSTANCE and
SHARED-INITIALIZE methods that are invoked by MAKE-INSTANCE.
Wrapping an (EVAL-WHEN (EVAL COMPILE LOAD) ...) around the appropriate
definitions will make sure they are fully defined at compile-time.
Alternatively, the definitions could be placed in a separate file,
which is loaded before compiling the file that depends on those
definitions.
Benefits:
Programmers can rely on programs that use the CLOS defining macros
being able to compile correctly in all implementations, without having
to wrap explicit EVAL-WHENs around every macro call.
Discussion:
Loosemore says:
Although I admit I don't understand all of the issues involved with
the meta-object protocol, I support proposal MINIMAL. I don't think
leaving the issue of whether or not classes can be instantiated at
compile-time unspecified places an undue burden on users, and it does
leave more freedom for the meta-object protocol to decide what the
right behavior really is.
-------
∂10-Mar-89 1324 CL-Compiler-mailer Issue: DEFINE-OPTIMIZER (Version 2)
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 10 Mar 89 13:24:11 PST
Received: from BOBOLINK.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 554754; Fri 10-Mar-89 16:21:48 EST
Date: Fri, 10 Mar 89 16:21 EST
From: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Issue: DEFINE-OPTIMIZER (Version 2)
To: CL-Compiler@SAIL.Stanford.EDU
cc: KMP@STONY-BROOK.SCRC.Symbolics.COM
Message-ID: <890310162127.0.KMP@BOBOLINK.SCRC.Symbolics.COM>
Most of the discussion which came from the previous draft came from the
following two points:
1. Some people were very confused by the use of STRING-APPEND as an
example, apparently because so many implementations offer
STRING-APPEND as an extension that everyone thought I was suggesting
that users could and should write optimizers for CL built-in functions,
something which I don't think is a good idea.
2. Some people wanted a pattern matching facility intertwined with this.
I addressed these problems in the following ways:
1. I removed the STRING-APPEND example and replaced it with a couple
of small examples taken directly from the Common Lisp Macsyma (R)
sources, since it was Macsyma that motivated this issue in the first
place. Macsyma actually has some considerably more elaborate
optimizers, but it would have been a lot more trouble for me to get
someone to authorize me to release the sources for them than for
these simple examples, so I hope these will suffice.
2. I added a note to the Discussion mentioning that this was a nice idea,
but that we don't have time to develop such a thing. However, as I
hope the examples illustrate, there is still quite a bit that can be
done even with the simpler facility proposed here, and I hope people
will seriously consider this without getting bogged down in a bunch
of really orthogonal issues.
-----
Issue: DEFINE-OPTIMIZER
References: None
Category: ADDITION
Edit history: 28-Sep-88, Version 1 by Pitman
10-Mar-89, Version 2 by Pitman (clarifications, new example)
Status: For Internal Discussion
Problem Description:
Often a general functional interface could be bypassed given explicit
knowledge of the arguments passed. This may happen when the arguments
are constant (or otherwise inferrable), an argument type is known (eg,
due to use of THE or DECLARE), or when some particular pattern of
optional, rest or keyword arguments is apparent.
Most implementations provide internally for optimization of generalized
function call interfaces to more specialized ones, but such an
optimization facility is not provided to Common Lisp users.
The absence of this facility in a portable fashion means that some
CL programs run slower than they need to in some implementations.
Proposal (DEFINE-OPTIMIZER:NEW-FACILITY):
Introduce a facility for declaring compiler optimizations.
DEFINE-OPTIMIZER name arglist &body forms
Defines a compiler optimizer for a function named NAME. The arglist
is treated exactly like the arglist to DEFMACRO, and may include
&ENVIRONMENT and &WHOLE.
When the optimizer is invoked, the forms are executed in the context
of bindings specified by the arglist, and two values are yielded,
RESULT and VALID-P. (If either of the first or second return value
is non-NIL, then the first return value is considered valid).
If the result is valid, it is a form which is preferrable to evaluate
instead of the indicated call.
OPTIMIZE-EXPRESSION-1 form env
Similar to MACROEXPAND-1. Invokes the optimizers for the top level of
FORM, but does not iterate on the result. Returns two values:
RESULT and CHANGED-P.
Note: If an optimizer returns a result which is not valid,
OPTIMIZE-EXPRESSION-1 hides the fact by returning FORM,NIL
rather than NIL,NIL.
OPTIMIZE-EXPRESSION form env
Iterates calling OPTIMIZE-EXPRESSION-1 until the CHANGED-P result
is NIL.
An implementation must save optimizer definitions created by
DEFINE-OPTIMIZER in case OPTIMIZE-EXPRESSION is attempted, but is
not actually required to call OPTIMIZE-EXPRESSION itself. Interpreters,
for example, may choose to just call the unoptimized form.
The effect of defining optimizations for functions on the LISP package
is not defined. (In some implementations, this would clobber or conflict
with existing advice that may be of higher quality.)
The editor is advised that a non-binding style note such as the
following would also be appropriate:
In general, it is poor style for a programmer to define optimizers for
functions that he does not maintain. This is because the correct
implementation of an optimizer for a function usually depends on an
understanding of the internals of that function. As such, a function
definition and any optimizers should be maintained as a unit so that
they can changes in either can be synchronized as appropriate with the
other.
Example:
;; These examples are taken literally from the Macsyma sources,
;; modified only to change DEFOPT to DEFINE-OPTIMIZER. The comments
;; were specially written for the X3J13 audience.
;; M+ is adds a Macsyma expression to another Macsyma expression.
;; The Macsyma internal representation for the sum of X and Y is
;; ((MPLUS) X Y). A all the real work is done by SIMPLIFY, which
;; reduces the expression as needed necessary. However, SIMPLIFY
;; is very complicated, and considerable speed can be gained by
;; entering it at specific known places.
(DEFUN M+ (&REST TERMS)
(PROTECT-&REST-VARIABLE TERMS)
(SIMPLIFY `((MPLUS) ,@TERMS)))
(DEFOPT M+ (&REST TERMS)
(COND ((= (LENGTH TERMS) 2) `(ADD2* ,@TERMS))
(T `(ADDN (LIST ,@TERMS) NIL))))
;; M- negates a Macsyma expression, or substracts two Macsyma
;; expressions. Once you figure out which of the two operations is
;; to be done, the problem is similar to that of M+ above. However,
;; often the decision can be made at compile time. In this case,
;; INLINE functions would have worked ok, except that not all
;; implementations do inlining, and even those that do may fail to
;; recognize that EXP2 being NIL means that a test can be eliminated
;; or dead code can be eliminated. Using optimizers is far more
;; likely to be useful in practice.
(DEFUN M- (EXP1 &OPTIONAL (EXP2 NIL EXP2P))
(IF (NOT EXP2P)
(M--INTERNAL-NEGATE EXP1)
(M--INTERNAL-SUBTRACT EXP1 EXP2)))
(DEFOPT M- (EXP1 &OPTIONAL (EXP2 NIL EXP2P))
(IF (NOT EXP2P)
`(M--INTERNAL-NEGATE ,EXP1)
`(M--INTERNAL-SUBTRACT ,EXP1 ,EXP2)))
Rationale:
Many large portable applications expect such a facility on an
implementation-specific basis. Others would use one if available.
Even if implementations don't use the provided optimizers primitively,
user macros and code-walkers can invoke them, so the facility wouldn't
be completely useless even in those implementations.
Current Practice:
Symbolics Genera provides an optimizer facility which is more elaborate
but not fundamentally incompatible with this facility.
Many (if not most) serious implementations provide a similar facility.
Cost to Implementors:
Since the implementation is not required to use this facility, the
cost of providing the proposed support is very small.
Cost to Users:
None. This change is upward compatible.
Cost of Non-Adoption:
Portable code would be slower than necessary in some situations.
Benefits:
Some existing non-portable code could become portable.
Aesthetics:
Providing a separate optimizer definition from a main function definition
makes a possibility that the optimizer and main function could drift out
of synch. However, most places where this gets used in the first place
are places where speed is of paramount importance and the programmer is
willing to invest effort in maintaining things correctly and to accept the
risk of lossage if s/he fails.
This is a fairly clean and simple extension which adds significant
power to the compiler.
Discussion:
Pitman strongly supports this proposal, the design of which is modeled
directly after that which has been used in Macsyma for many years.
Information about argument type can come from two different sources:
THE and declarations (via PROCLAIM or DECLARE). The former information
is portably accessible, the latter is not. While a separate proposal
for allowing program access to type declarations would be make this
facility more useful, it is still quite useful without it, as the
examples from Macsyma illustrate.
Some implementations provide a way to provide more than one optimizer
for the same function. A multiple optimizer facility can be written
in terms of this simpler facility and vice versa, so the simpler of
the two facilities is proposed here.
Some people have suggested that they would like to see a pattern
matching facility integrated into this facility. The design of a
facility that would satisfy everyone would take a lot of time and
effort. At this point, there is no chance that the design of such a
facility would occur in time for acceptance into the standard.
The choice is this or nothing. Pitman thinks the language is much
better off with some form of optimization support than none.
∂10-Mar-89 1407 CL-Compiler-mailer Issue: WITH-COMPILATION-UNIT (Version 2)
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 10 Mar 89 14:07:20 PST
Received: from BOBOLINK.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 554788; Fri 10-Mar-89 17:04:58 EST
Date: Fri, 10 Mar 89 17:04 EST
From: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Issue: WITH-COMPILATION-UNIT (Version 2)
To: CL-Compiler@SAIL.Stanford.EDU
Message-ID: <890310170444.1.KMP@BOBOLINK.SCRC.Symbolics.COM>
I read back over the mail related to this and most of it was centered
around extensions that we might want to do. I think we can't really
consider most of those extensions at this point because it's too late in
the game.
Basically, I merged the few comments that were easy to do without
disturbing the simple nature of the proposal.
I think, however, that if we can just get behind this straightforward,
extensible proposal, there will be a good framework in place for
extending it in some subsequent standard after individual
implementations have had time to experiment a bit with the paradigm.
-----
Issue: WITH-COMPILATION-UNIT
References: COMPILE (p438), COMPILE-FILE (p439)
Category: ADDITION
Edit history: 29-Sep-88, Version 1 by Pitman
10-Mar-89, Version 2 by Pitman (merge comments)
Status: For Internal Discussion
Problem Description:
Some actions done by the compiler (and particularly the file compiler)
are typically deferred until the "very end" of compilation. For example,
some compilers complain about "functions seen but not defined".
Unfortunately, since COMPILE-FILE is the largest unit of granularity,
and since systems often extend over more than one file, it often happens
that the compiler needlessly complains at the end of compiling one file
about the absence of functions defined in the next file.
Proposal (WITH-COMPILATION-UNIT:NEW-MACRO):
Add the following new macro:
WITH-COMPILATION-UNIT options &BODY forms [Macro]
Executes forms from left to right. Within the dynamic context
of this form, actions deferred by the compiler until "the end of
compilation" will be deferred until the end of the outermost call
to WITH-COMPILATION-UNIT. The result(s) are the same as that of
the last of the FORMS (or NIL if FORMS is null).
OPTIONS is a keyword/value list, where only the values are
evaluated. The set of keywords permitted may be extended by the
implementation, but the only keyword defined by this standard is:
:OVERRIDE boolean
The default is NIL. If nested dynamically only the outer call
to WITH-COMPILATION-UNIT has any effect unless BOOLEAN is T,
in which case warnings are deferred only to the end of the
innermost call.
It follows that the functions COMPILE and COMPILE-FILE should
provide the effect of (WITH-COMPILATION-UNIT (:OVERRIDE NIL) ...)
around their code.
Note also that not all warnings are deferred. In some implementations,
it may be that none are deferred. This proposal only creates an
interface to the capability where it exists, it does not require the
creation of the capability. An implementation which does not do
deferred warnings may correctly implement this as expanding into PROGN.
Test Case:
(DEFUN COMPILE-FILES (&REST FILES)
(WITH-COMPILATION-UNIT ()
(MAPCAR #'(LAMBDA (FILE) (COMPILE-FILE FILE)) FILES)))
(COMPILE-FILES "A" "B" "C")
processes deferred warnings only after compiling all of A, B, and C.
Rationale:
This will make the development of portable system-construction tools
considerably more convenient.
Current Practice:
Lucid has a very similar facility, called WITH-DEFERRED-WARNINGS.
TI Explorer and Symbolics Genera have a similar facility, which they
call COMPILER-WARNING-CONTEXT-BIND.
Cost to Implementors:
In implementations which have no deferred warnings, there is no cost.
In implementations which have deferred warnings, the cost is probably
fairly small -- usually just a matter of writing interfacing the
proposed macro to an existing one.
Cost to Users:
None. This is a compatible addition.
Cost of Non-Adoption:
Portable system-construction tools would continue to print lots of
spurious warnings because they would have no way to tell the system
that a set of files was working together.
Benefits:
The cost of non-adoption is avoided.
Aesthetics:
The ability to create a compilation unit other than a file is important.
Discussion:
Pitman and Benson support this addition.
One could imagine adding more options at a later date.
It was the opinion of the compiler committee that there was room for
expansion here to address issues like bounding the scope of global
proclamations, sharing compile-time environments across files, etc.
However, insufficient work was done on this to justify putting such
a thing into the standard. The only clear need we have at this time
was to defer warnings, but we chose a general name like
WITH-COMPILATION-UNIT rather than a specific name like Lucid's
WITH-DEFERRED-WARNINGS in order to encourage implementations to
experiment with other kinds of options under implementation-specific
keywords. Perhaps by the time of the next standard there will be
sufficient understanding of this area to warrant further elaboration
of this primitive.
∂10-Mar-89 1525 CL-Compiler-mailer Re: Issue: DEFINE-OPTIMIZER (Version 2)
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 10 Mar 89 15:25:21 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA11643; Fri, 10 Mar 89 16:22:57 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA11082; Fri, 10 Mar 89 16:22:55 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903102322.AA11082@defun.utah.edu>
Date: Fri, 10 Mar 89 16:22:53 MST
Subject: Re: Issue: DEFINE-OPTIMIZER (Version 2)
To: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>
Cc: CL-Compiler@SAIL.Stanford.EDU
In-Reply-To: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>, Fri, 10 Mar 89 16:21 EST
This is looking better, but I personally would still want to see
a few things clarified before I could vote for it.
o Can you put declarations in the body? I don't see any reason why not,
except that the writeup doesn't say so.
o Can you define an optimizer for a macro as well as a function? (If it
were permitted, it might make the interaction between OPTIMIZE-EXPRESSION
and MACROEXPAND kind of confusing.)
o I don't think OPTIMIZE-EXPRESSION-1 should apply an optimizer that
has been defined for a function that is lexically shadowed (as by
an FLET or MACROLET) in the environment. Can we say something
specific about what happen in this situation, one way or the other?
Otherwise, I think the basic idea is fine.
-Sandra
-------
∂10-Mar-89 1556 CL-Compiler-mailer Re: Issue: WITH-COMPILATION-UNIT (Version 2)
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 10 Mar 89 15:56:12 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA12911; Fri, 10 Mar 89 16:54:00 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA11110; Fri, 10 Mar 89 16:53:29 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903102353.AA11110@defun.utah.edu>
Date: Fri, 10 Mar 89 16:53:27 MST
Subject: Re: Issue: WITH-COMPILATION-UNIT (Version 2)
To: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>
Cc: CL-Compiler@SAIL.Stanford.EDU
In-Reply-To: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>, Fri, 10 Mar 89 17:04 EST
This looks generally OK. One question I have is whether
implementations may extend this construct to do things other than
handle deferred warnings, without requiring the user to explicitly
provide an option to get that behavior. (Sharing of compilation
environments is a good example.)
As an editorial nit, I'd like to see something in the proposal itself
to indicate that the purpose of WITH-COMPILATION-UNIT is more general
than just causing warnings to be deferred. This really goes
hand-in-hand with the above -- if we say that the purpose of
WITH-COMPILATION-UNIT is to tell the compiler that it should treat
multiple calls as relating to the same module or unit, then I think it
would be OK for an implementation to do other things as well. On the
other hand, if we say that the purpose is only to defer warnings, then
the only thing it should do by default is defer warnings.
-Sandra
-------
∂10-Mar-89 1553 CL-Cleanup-mailer Re: issue IN-PACKAGE-FUNCTIONALITY, version 7
Received: from Xerox.COM by SAIL.Stanford.EDU with TCP; 10 Mar 89 15:53:15 PST
Received: from Semillon.ms by ArpaGateway.ms ; 10 MAR 89 15:50:29 PST
Date: 10 Mar 89 15:49 PST
From: masinter.pa@Xerox.COM
Subject: Re: issue IN-PACKAGE-FUNCTIONALITY, version 7
In-reply-to: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>'s message
of Fri, 10 Mar 89 12:55 EST
To: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
cc: Sandra J Loosemore <sandra%defun@cs.utah.edu>,
cl-cleanup@sail.stanford.edu, cl-compiler@sail.stanford.edu
Message-ID: <890310-155029-16287@Xerox>
maybe I'm late, but I still liked SELECT-ONLY better than NEW-MACRO. Just
to calibrate, I didn't like removing REQUIRE & PROVIDE, either.
∂10-Mar-89 1615 CL-Compiler-mailer Re: Issue: WITH-COMPILATION-UNIT (Version 2)
Received: from ALDERAAN.SCRC.Symbolics.COM ([128.81.41.109]) by SAIL.Stanford.EDU with TCP; 10 Mar 89 16:15:44 PST
Received: from BOBOLINK.SCRC.Symbolics.COM by ALDERAAN.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 280423; Fri 10-Mar-89 19:12:36 EST
Date: Fri, 10 Mar 89 19:12 EST
From: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Re: Issue: WITH-COMPILATION-UNIT (Version 2)
To: Sandra%defun@cs.utah.edu
cc: CL-Compiler@SAIL.Stanford.EDU
In-Reply-To: <8903102353.AA11110@defun.utah.edu>
Message-ID: <890310191241.7.KMP@BOBOLINK.SCRC.Symbolics.COM>
My belief was that if it does anything other than defer warnings without
your having to ask it to (using keywords), then portability would
be sacrificed because it may have radically different effects in
each implementation. On the next standardization, we can talk
about not only adding options, but allowing them to default.
I guess it should say in the Proposal part:
Any implementation-dependent extensions may only be provided
as the result of an explicit programmer request by use of
an implementation-dependent keyword. Implementations are forbidden
from attaching additional meaning to a conforming use of this
macro.
If we just said that, would that alleviate the need for further examples?
∂10-Mar-89 1730 CL-Compiler-mailer issue DEFINE-OPTIMIZER, version 3
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 10 Mar 89 17:29:53 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA14983; Fri, 10 Mar 89 18:27:46 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA11236; Fri, 10 Mar 89 18:27:41 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903110127.AA11236@defun.utah.edu>
Date: Fri, 10 Mar 89 18:27:40 MST
Subject: issue DEFINE-OPTIMIZER, version 3
To: cl-compiler@sail.stanford.edu
Kent and I have given this one another iteration to fix the
ambiguities. Hopefully we've caught all the bugs this time.
Forum: Compiler
Issue: DEFINE-OPTIMIZER
References: Issue SYNTACTIC-ENVIRONMENT-ACCESS
Category: ADDITION
Edit history: 28-Sep-88, Version 1 by Pitman
10-Mar-89, Version 2 by Pitman (clarifications, new example),
10-Mar-89, Version 3 by Pitman & Loosemore
Status: Ready for release
Problem Description:
Often a general functional interface could be bypassed given explicit
knowledge of the arguments passed. This may happen when the arguments
are constant (or otherwise inferrable), an argument type is known (eg,
due to use of THE or DECLARE), or when some particular pattern of
optional, rest or keyword arguments is apparent.
Most implementations provide internally for optimization of generalized
function call interfaces to more specialized ones, but such an
optimization facility is not provided to Common Lisp users.
The absence of this facility in a portable fashion means that some
CL programs run slower than they need to in some implementations.
Proposal (DEFINE-OPTIMIZER:NEW-FACILITY):
Introduce a facility for declaring compiler optimizations.
DEFINE-OPTIMIZER name arglist {declaration}* {form}* [Macro]
Defines a compiler optimizer for a function named NAME. The ARGLIST,
DECLARATIONS, and FORMS are treated exactly like the arglist,
declarations, and forms in a DEFMACRO. (The arglist may include
&ENVIRONMENT and &WHOLE.)
The argument NAME must name a function which has been previously
defined. The effects of defining an optimizer for a locally or
globally defined macro, a locally defined function, or a special
form are undefined.
When the optimizer is invoked, the forms are executed in the context
of bindings specified by the arglist, and two values are yielded,
RESULT and VALID-P. (If either of the first or second return value
is non-NIL, then the first return value is considered valid).
If the result is valid, it is a form which is preferrable to evaluate
instead of the indicated call.
If a call to DEFINE-OPTIMIZER appears at top-level in a file
being processed by the file compiler, it also makes the optimizer
known at compile-time (similar to the way DEFMACRO makes a macro
definition known to the compiler).
OPTIMIZE-EXPRESSION-1 form env [Function]
Similar to MACROEXPAND-1. Invokes the optimizers for the top level of
FORM, but does not iterate on the result. Returns two values:
RESULT and CHANGED-P.
Note: If an optimizer returns a result which is not valid,
OPTIMIZE-EXPRESSION-1 hides the fact by returning FORM,NIL
rather than NIL,NIL.
OPTIMIZE-EXPRESSION form env [Function]
Iterates calling OPTIMIZE-EXPRESSION-1 until the CHANGED-P result
is NIL.
An implementation must save optimizer definitions created by
DEFINE-OPTIMIZER in case OPTIMIZE-EXPRESSION is attempted, but is
not actually required to call OPTIMIZE-EXPRESSION itself. Interpreters,
for example, may choose to just call the unoptimized form.
Using FLET and MACROLET shadow not only functions and their SETF methods,
but also their optimizers. No portable facility is provided for creating
locally defined optimizers.
The effect of defining optimizations for functions on the LISP package
is not defined. (In some implementations, this would clobber or conflict
with existing advice that may be of higher quality.)
The editor is advised that a non-binding style note such as the
following would also be appropriate:
In general, it is poor style for a programmer to define optimizers for
functions that he does not maintain. This is because the correct
implementation of an optimizer for a function usually depends on an
understanding of the internals of that function. As such, a function
definition and any optimizers should be maintained as a unit so that
they can changes in either can be synchronized as appropriate with the
other.
Example:
;; These examples are taken literally from the Macsyma sources,
;; modified only to change DEFOPT to DEFINE-OPTIMIZER. The comments
;; were specially written for the X3J13 audience.
;; M+ is adds a Macsyma expression to another Macsyma expression.
;; The Macsyma internal representation for the sum of X and Y is
;; ((MPLUS) X Y). A all the real work is done by SIMPLIFY, which
;; reduces the expression as needed necessary. However, SIMPLIFY
;; is very complicated, and considerable speed can be gained by
;; entering it at specific known places.
(DEFUN M+ (&REST TERMS)
(PROTECT-&REST-VARIABLE TERMS)
(SIMPLIFY `((MPLUS) ,@TERMS)))
(DEFINE-OPTIMIZER M+ (&REST TERMS)
(COND ((= (LENGTH TERMS) 2) `(ADD2* ,@TERMS))
(T `(ADDN (LIST ,@TERMS) NIL))))
;; M- negates a Macsyma expression, or substracts two Macsyma
;; expressions. Once you figure out which of the two operations is
;; to be done, the problem is similar to that of M+ above. However,
;; often the decision can be made at compile time. In this case,
;; INLINE functions would have worked ok, except that not all
;; implementations do inlining, and even those that do may fail to
;; recognize that EXP2 being NIL means that a test can be eliminated
;; or dead code can be eliminated. Using optimizers is far more
;; likely to be useful in practice.
(DEFUN M- (EXP1 &OPTIONAL (EXP2 NIL EXP2P))
(IF (NOT EXP2P)
(M--INTERNAL-NEGATE EXP1)
(M--INTERNAL-SUBTRACT EXP1 EXP2)))
(DEFINE-OPTIMIZER M- (EXP1 &OPTIONAL (EXP2 NIL EXP2P))
(IF (NOT EXP2P)
`(M--INTERNAL-NEGATE ,EXP1)
`(M--INTERNAL-SUBTRACT ,EXP1 ,EXP2)))
Rationale:
Many large portable applications expect such a facility on an
implementation-specific basis. Others would use one if available.
Even if implementations don't use the provided optimizers primitively,
user macros and code-walkers can invoke them, so the facility wouldn't
be completely useless even in those implementations.
Current Practice:
Symbolics Genera provides an optimizer facility which is more elaborate
but not fundamentally incompatible with this facility.
Many (if not most) serious implementations provide a similar facility.
For example, Lucid provides "compiler macros" which serve the same
purpose.
Cost to Implementors:
Since the implementation is not required to use this facility, the
cost of providing the proposed support is very small.
Cost to Users:
None. This change is upward compatible.
Cost of Non-Adoption:
Portable code would be slower than necessary in some situations.
Benefits:
Some existing non-portable code could become portable.
Aesthetics:
Providing a separate optimizer definition from a main function definition
makes a possibility that the optimizer and main function could drift out
of synch. However, most places where this gets used in the first place
are places where speed is of paramount importance and the programmer is
willing to invest effort in maintaining things correctly and to accept the
risk of lossage if s/he fails.
This is a fairly clean and simple extension which adds significant
power to the compiler.
Discussion:
Pitman strongly supports this proposal, the design of which is modeled
directly after that which has been used in Macsyma for many years.
Information about argument type can come from two different sources:
THE and declarations (via PROCLAIM or DECLARE). The former information
is portably accessible, the latter is not. While a separate proposal
(SYNTACTIC-ENVIRONMENT-ACCESS) for allowing program access to type
declarations would be make this facility more useful, it is still
quite useful without it, as the examples from Macsyma illustrate.
Some implementations provide a way to provide more than one optimizer
for the same function. A multiple optimizer facility can be written
in terms of this simpler facility and vice versa, so the simpler of
the two facilities is proposed here.
Some people have suggested that they would like to see a pattern
matching facility integrated into this facility. The design of a
facility that would satisfy everyone would take a lot of time and
effort. At this point, there is no chance that the design of such a
facility would occur in time for acceptance into the standard.
The choice is this or nothing. Pitman thinks the language is much
better off with some form of optimization support than none.
Loosemore says:
Although I don't really think this is an essential feature to include
in the standard, I don't have any strong objection to adding it. If
people think it's a good idea to provide a standard interface for this
kind of thing, this is a good proposal for doing it -- it's fairly
simple, doesn't introduce any radically new ideas, and is general
enough to allow alternate interfaces (such as the pattern matcher) to
be layered on top of it.
-------
∂11-Mar-89 0126 Common-Lisp-Object-System-mailer Re: issue CLOS-MACRO-COMPILATION, version 1
Received: from ti.com by SAIL.Stanford.EDU with TCP; 11 Mar 89 01:25:50 PST
Received: by ti.com id AA05982; Fri, 10 Mar 89 18:21:46 CST
Received: from Kelvin by tilde id AA27610; Fri, 10 Mar 89 18:07:29 CST
Message-Id: <2814566813-5018117@Kelvin>
Sender: GRAY@Kelvin.csc.ti.com
Date: Fri, 10 Mar 89 18:06:53 CST
From: David N Gray <Gray@DSG.csc.ti.com>
To: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Cc: cl-compiler@sail.stanford.edu, common-lisp-object-system@sail.stanford.edu
Subject: Re: issue CLOS-MACRO-COMPILATION, version 1
In-Reply-To: Msg of Fri, 10 Mar 89 13:56:05 MST from sandra%defun@cs.utah.edu (Sandra J Loosemore)
> Proposal CLOS-MACRO-COMPILATION:MINIMAL:
...
> DEFMETHOD:
>
> * The method is not callable at compile-time. If there is a generic
> function with the same name at compile-time, compiling a DEFMETHOD
> will not add the method to that generic function.
>
> [This also seems to imply that tests for existence of the generic
> function, lambda-list congruence, etc. must not happen until
> load time.]
No, an implementation should be permitted to check for lambda-list
congruence between methods defined in the same file; this doesn't
require any reference to the resident generic function definition. If
the file doesn't include a DEFGENERIC, then the first DEFMETHOD defines
the compile-time generic function attributes, and subsequent methods can
be checked against that.
> DEFINE-METHOD-COMBINATION:
>
> * The method combination can be used in a subsequent DEFGENERIC. If it
> is referenced, the body of a long form of method combination must be
> evaluable at compile-time.
But if methods are not installed at compile time and generic functions
are not callable at compile time, then I don't think there is any
situation in which the method combination body could be executed at
compile-time.
∂11-Mar-89 0152 CL-Compiler-mailer Re: issue MACRO-ENVIRONMENT-EXTENT, version 2
Received: from ti.com by SAIL.Stanford.EDU with TCP; 11 Mar 89 01:51:05 PST
Received: by ti.com id AA04610; Fri, 10 Mar 89 11:48:18 CST
Received: from Kelvin by tilde id AA18376; Fri, 10 Mar 89 11:31:11 CST
Message-Id: <2814543045-3466116@Kelvin>
Sender: GRAY@Kelvin.csc.ti.com
Date: Fri, 10 Mar 89 11:30:45 CST
From: David N Gray <Gray@DSG.csc.ti.com>
To: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Cc: cl-compiler@sail.stanford.edu,
"David A. Moon" <Moon@STONY-BROOK.SCRC.Symbolics.COM>,
"Kim A. Barrett" <IIM%ECLA@ECLC.USC.EDU>
Subject: Re: issue MACRO-ENVIRONMENT-EXTENT, version 2
In-Reply-To: Msg of Thu, 9 Mar 89 15:18:33 MST from sandra%defun@cs.utah.edu (Sandra J Loosemore)
I think there are still too many unresolved relationships for this issue to
be ready for voting. We have talked about using environments for three
different things, each of which has different implementation
considerations that affect the extent:
1. Macro definitions
If this were all that environments were used for, then proposal
MACRO-ENVIRONMENT-EXTENT:INDEFINITE would be fine.
2. Information about current variable bindings, local functions, type
declarations, and OPTIMIZE declarations (as in issue
SYNTACTIC-ENVIRONMENT-ACCESS).
I don't want to have to support access to this information from a
saved environment object after the compiler has moved on to a
different lexical scope. Proposal
MACRO-ENVIRONMENT-EXTENT:DYNAMIC would solve that, but is more
restrictive than it needs to be.
3. Class definitions.
For these, we want the extent to end at the conclusion of
COMPILE-FILE. Again, proposal MACRO-ENVIRONMENT-EXTENT:DYNAMIC is
much more restrictive than necessary.
I have been talking here about the semantic meaningfulness of the
environments, but there is an orthogonal issue of data representation. If
the concern is that the environment could be a stack list which physically
no longer exists outside its dynamic extent, then proposal
MACRO-ENVIRONMENT-EXTENT:DYNAMIC-WITH-COPIER makes sense as a way to copy
the list to the heap. The copier does not appear to be useful for
addressing the questions of semantic extent above.
How about the following:
Proposal MACRO-ENVIRONMENT-EXTENT:EXTENDED-DYNAMIC-WITH-COPIER:
State that macro environment objects received with the &ENVIRONMENT
argument of a macro function or with a *MACROEXPAND-HOOK* function
have only dynamic extent; the consequences are undefined if they are
referred to outside the dynamic extent of that macro function or hook
function.
Add a new function:
COPY-ENVIRONMENT environment [function]
This function returns an environment object that is semantically
equivalent to "environment" (which must be an object of the type
received with an &ENVIRONMENT argument to a macro or as an argument to
a *MACROEXPAND-HOOK* function), but which may safely be referred to
outside the dynamic extent of the macro function. This function is
permitted to return an object that is EQ to its argument if that
object may be safely used.
The meaningfulness of the copied environment is still subject to the
limitation that it can't be used to access information after the processor
which created the environment (such as a compiler or interpreter) is no
longer processing within the scope of that information. Thus, the
environment can be used in a macro expansion, since the expansion will be
processed in the same lexical scope. Declarations, variable bindings, and
local function and macro definitions [assuming passage of proposal
SYNTACTIC-ENVIRONMENT-ACCESS] can not be reliably accessed at a later
time. Within COMPILE-FILE, class definitions resulting from a top-level
DEFCLASS can not be accessed after processing of the file is complete.
Rationale:
This allows implementations to use somewhat more efficient techniques
for representing environment objects. For example, the storage could
be stack-allocated, or environments could be bashed destructively
instead of always being freshly heap-allocated.
It also allows programmers to use idioms that rely on using environment
objects outside of the macro expander, while defining limitations on the
usable extent of that information. These semantic limitations provide
flexibility in how the implementation actually represents environments,
which is important for minimizing the need to change existing
implementations.
∂11-Mar-89 0200 CL-Compiler-mailer Re: Issue: DEFINE-OPTIMIZER (Version 2)
Received: from Sun.COM by SAIL.Stanford.EDU with TCP; 11 Mar 89 02:00:50 PST
Received: from snail.Sun.COM by Sun.COM (4.1/SMI-4.0)
id AA11684; Sat, 11 Mar 89 02:01:37 PST
Received: from clam.sun.com by snail.Sun.COM (4.1/SMI-4.0)
id AA24334; Fri, 10 Mar 89 14:31:38 PST
Received: by clam.sun.com (4.0/SMI-4.0)
id AA15806; Fri, 10 Mar 89 14:35:06 PST
Date: Fri, 10 Mar 89 14:35:06 PST
From: cperdue@Sun.COM (Cris Perdue)
Message-Id: <8903102235.AA15806@clam.sun.com>
To: CL-Compiler@SAIL.Stanford.EDU, KMP@STONY-BROOK.SCRC.Symbolics.COM
Subject: Re: Issue: DEFINE-OPTIMIZER (Version 2)
Optimizers are important in my experience also. In fact, I would
have sat down on Saturday to write a similar proposal had Kent
not distributed this one.
I support Kent's proposal, with the following suggestions:
An additional function that corresponds to MACRO-FUNCTION for ordinary
macros. I include the proposed definition for this, which follows the
definition of MACRO-FUNCTION in CLtL, p144. This provides
primitive-level access, including the ability to undefine an
optimizer.
Add to the problem description that sometimes an "operator" is defined
as a macro purely because its performance would be inadequate if
implemented as a function.
Also, I believe it would be best to state that in environments where
the compiler is prohibited from treating a function as INLINE,
optimizers are deactivated. In particular, the functions
OPTIMIZE-EXPRESSION and OPTIMIZE-EXPRESSION-1 do not transform their
expression argument if given an environment in which the function must
not be treated as INLINE. (I recognize that this is a bit of
a judgment call.)
The point here is to make user-defined optimizers follow the behavior
of optimizers built-in to compilers in their response to NOTINLINE.
OPTIMIZER-FUNCTION symbol
This function corresponds directly to MACRO-FUNCTION for macros.
The argument must be a symbol. If the symbol has an optimizer
defined, then the optimizer function is returned. An optimizer
function takes two arguments, the expression that may be
optimized and an environment of the same kind accepted by macro
expansion functions. If the symbol has no optimizer, then NIL is
returned.
SETF may be used with OPTIMIZER-FUNCTION to install or change an
optimizer. If NIL is given as the new value, any existing optimizer
definition is removed.
∂11-Mar-89 0150 Common-Lisp-Object-System-mailer Re: Issue MACRO-ENVIRONMENT-EXTENT
Received: from ti.com by SAIL.Stanford.EDU with TCP; 11 Mar 89 01:48:49 PST
Received: by ti.com id AA04514; Fri, 10 Mar 89 11:03:22 CST
Received: from Kelvin by tilde id AA17547; Fri, 10 Mar 89 10:49:22 CST
Message-Id: <2814540528-3314903@Kelvin>
Sender: GRAY@Kelvin.csc.ti.com
Date: Fri, 10 Mar 89 10:48:48 CST
From: David N Gray <Gray@DSG.csc.ti.com>
To: "David A. Moon" <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Cc: "Kim A. Barrett" <IIM%ECLA@ECLC.USC.EDU>, sandra%defun@CS.UTAH.EDU,
cl-compiler@SAIL.STANFORD.EDU,
common-lisp-object-system@SAIL.STANFORD.EDU
Subject: Re: Issue MACRO-ENVIRONMENT-EXTENT
In-Reply-To: Msg of Thu, 9 Mar 89 13:59 EST from David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
> The extent of macro environment objects is related to EVAL-WHEN because macro
> expanders may wish to return forms which contain environments as quoted
> constants.
>
> I am convinced that this should be ruled out, and that CLOS made a mistake
> here. (Incidentally the part of CLOS that says this is in chapter 3, the
> accepted part of CLOS does not say anything about the expansion of the
> macros is.)
Except for ENSURE-GENERIC-FUNCTION, which is defined in chapter 2 with an
:ENVIRONMENT argument, and referenced on page 2-28 as part of the
semantics of DEFGENERIC. Maybe this function should have been in chapter
3 instead? Should it be removed from the standard?
> Requiring environments to have indefinite extent has
> problems for CLOS because at compile-time it wants to create remote metaobjects
> and link them into the right places, but then flush those links when the
> compilation is over.
>
> This depends on whether you think the environment actually contains the
> table that relates names to objects, or just contains a boolean flag
> that tells functions such as FIND-CLASS which of two tables to look in.
> Under the latter model, nothing about the environment prevents the
> COMPILE-FILE table from being reset at any time. This is one reason
> why I think the second model is right.
But if you reset the table, then a FIND-CLASS on that environment will no
longer be meaningful, which has the same effect as being outside the valid
extent of the environment.
∂11-Mar-89 0810 CL-Compiler-mailer Re: issue MACRO-ENVIRONMENT-EXTENT, version 2
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 11 Mar 89 08:09:16 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA03663; Sat, 11 Mar 89 09:05:03 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA00528; Sat, 11 Mar 89 09:03:44 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903111603.AA00528@defun.utah.edu>
Date: Sat, 11 Mar 89 09:03:42 MST
Subject: Re: issue MACRO-ENVIRONMENT-EXTENT, version 2
To: David N Gray <Gray@DSG.csc.ti.com>
Cc: sandra%defun@cs.utah.edu (Sandra J Loosemore),
cl-compiler@sail.stanford.edu,
"David A. Moon" <Moon@STONY-BROOK.SCRC.Symbolics.COM>,
"Kim A. Barrett" <IIM%ECLA@ECLC.USC.EDU>
In-Reply-To: David N Gray <Gray@DSG.csc.ti.com>, Fri, 10 Mar 89 11:30:45 CST
> Date: Fri, 10 Mar 89 11:30:45 CST
> From: David N Gray <Gray@DSG.csc.ti.com>
>
> Proposal MACRO-ENVIRONMENT-EXTENT:EXTENDED-DYNAMIC-WITH-COPIER:
>
> [...]
>
> The meaningfulness of the copied environment is still subject to the
> limitation that it can't be used to access information after the processor
> which created the environment (such as a compiler or interpreter) is no
> longer processing within the scope of that information. Thus, the
> environment can be used in a macro expansion, since the expansion will be
> processed in the same lexical scope. Declarations, variable bindings, and
> local function and macro definitions [assuming passage of proposal
> SYNTACTIC-ENVIRONMENT-ACCESS] can not be reliably accessed at a later
> time.
This is too vague. What does "scope of that information" mean? Dynamic
scope? Lexical scope? You need to say what the *extent* is, not the
scope.
An example may clarify my confusion. :-) Suppose I'm using
LOCAL-TYPE-DECLARE as implemented in the writeup (but with
COPY-ENVIRONMENT added to it, of course), and then I say something
like:
(defun f (x)
(local-type-declare ((x fixnum))
#'(lambda (y)
(local-type-declare ((y fixnum))
(+ (typed var x) y)))
))
Is this legitimate? Let's assume that the interpreter doesn't do a
code walk. The closure returned when you call F has indefinite
extent, yet it contains a reference to the environment established by
the lexically enclosing LOCAL-TYPE-DECLARE. So if this is legal,
the environment has to have indefinite extent too.
Rather than trying to add all these ill-defined qualifications and
restrictions, if we absolutely cannot agree to give environments
indefinite extent (or to provide a copier to give them indefinite
extent), I would much rather see a simple statement that you can't
assume more than dynamic extent.
> Within COMPILE-FILE, class definitions resulting from a top-level
> DEFCLASS can not be accessed after processing of the file is complete.
As far as I've been able to tell, there is nothing in CLOS chapters 1
& 2 that says that DEFCLASS *must* store class definitions in any
particular way that involves environment objects. I think it would be
premature to say anything about this here until the meta-object protocol
settles down. Also, we don't have any similar restrictions on DEFMACRO
or whatever.
-Sandra
-------
∂11-Mar-89 0818 Common-Lisp-Object-System-mailer Re: issue CLOS-MACRO-COMPILATION, version 1
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 11 Mar 89 08:17:53 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA03974; Sat, 11 Mar 89 09:15:41 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA00548; Sat, 11 Mar 89 09:15:39 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903111615.AA00548@defun.utah.edu>
Date: Sat, 11 Mar 89 09:15:37 MST
Subject: Re: issue CLOS-MACRO-COMPILATION, version 1
To: David N Gray <Gray@DSG.csc.ti.com>
Cc: sandra%defun@cs.utah.edu (Sandra J Loosemore),
cl-compiler@sail.stanford.edu,
common-lisp-object-system@sail.stanford.edu
In-Reply-To: David N Gray <Gray@DSG.csc.ti.com>, Fri, 10 Mar 89 18:06:53 CST
> Date: Fri, 10 Mar 89 18:06:53 CST
> From: David N Gray <Gray@DSG.csc.ti.com>
>
> > [This also seems to imply that tests for existence of the generic
> > function, lambda-list congruence, etc. must not happen until
> > load time.]
>
> No, an implementation should be permitted to check for lambda-list
> congruence between methods defined in the same file; this doesn't
> require any reference to the resident generic function definition. If
> the file doesn't include a DEFGENERIC, then the first DEFMETHOD defines
> the compile-time generic function attributes, and subsequent methods can
> be checked against that.
The description of DEFMETHOD in CLOS chapter 2 talks about calling
FBOUNDP and signalling an error if the function is not a generic
function, or if it is a generic function but the lambda list of the
method is not congruent. Clearly this shouldn't happen at
compile-time. I agree that the behavior you suggest makes more sense.
>
> > DEFINE-METHOD-COMBINATION:
> >
> > * The method combination can be used in a subsequent DEFGENERIC. If it
> > is referenced, the body of a long form of method combination must be
> > evaluable at compile-time.
>
> But if methods are not installed at compile time and generic functions
> are not callable at compile time, then I don't think there is any
> situation in which the method combination body could be executed at
> compile-time.
This is something I couldn't quite figure out from reading chapters 1
& 2. At what time does the method combination become "integrated"
into the DEFGENERIC? Does the process of expanding the DEFGENERIC
macro capture the method combination definition, in the same way that
expanding a SETF macro captures the setf method? Or does this happen
when you actually execute the DEFGENERIC macro?
-Sandra
-------
∂11-Mar-89 1220 Common-Lisp-Object-System-mailer Re: issue CLOS-MACRO-COMPILATION, version 1
Received: from ti.com by SAIL.Stanford.EDU with TCP; 11 Mar 89 12:19:38 PST
Received: by ti.com id AA10411; Sat, 11 Mar 89 14:18:09 CST
Received: from Kelvin by tilde id AA14567; Sat, 11 Mar 89 14:14:39 CST
Message-Id: <2814639239-9369595@Kelvin>
Sender: GRAY@Kelvin.csc.ti.com
Date: Sat, 11 Mar 89 14:13:59 CST
From: David N Gray <Gray@DSG.csc.ti.com>
To: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Cc: cl-compiler@sail.stanford.edu, common-lisp-object-system@sail.stanford.edu
Subject: Re: issue CLOS-MACRO-COMPILATION, version 1
In-Reply-To: Msg of Sat, 11 Mar 89 09:15:37 MST from sandra%defun@cs.utah.edu (Sandra J Loosemore)
> The description of DEFMETHOD in CLOS chapter 2 talks about calling
> FBOUNDP and signalling an error if the function is not a generic
> function, or if it is a generic function but the lambda list of the
> method is not congruent. Clearly this shouldn't happen at
> compile-time.
Right, unless it is viewed as doing the FBOUNDP in the compile-time
environment without inheritance from the resident environment.
> > But if methods are not installed at compile time and generic functions
> > are not callable at compile time, then I don't think there is any
> > situation in which the method combination body could be executed at
> > compile-time.
>
> This is something I couldn't quite figure out from reading chapters 1
> & 2. At what time does the method combination become "integrated"
> into the DEFGENERIC? Does the process of expanding the DEFGENERIC
> macro capture the method combination definition, in the same way that
> expanding a SETF macro captures the setf method? Or does this happen
> when you actually execute the DEFGENERIC macro?
This is only in chapter 3, and not very clear there even. My
understanding of it is that the method combination body would be invoked
from COMPUTE-EFFECTIVE-METHOD, which, depending on the implementation,
could be invoked as soon as a call to ADD-METHOD, or as late as a call
to the generic function which actually needs that particular
combination. Since it operates on a list of applicable methods, it
couldn't be invoked from DEFGENERIC. The generic function object would
just have a pointer to the method combination object for future use.
∂11-Mar-89 1243 CL-Compiler-mailer issue PROCLAIM-ETC-IN-COMPILE-FILE, version 3
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 11 Mar 89 12:43:42 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA07469; Sat, 11 Mar 89 13:41:29 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA00673; Sat, 11 Mar 89 13:41:26 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903112041.AA00673@defun.utah.edu>
Date: Sat, 11 Mar 89 13:41:24 MST
Subject: issue PROCLAIM-ETC-IN-COMPILE-FILE, version 3
To: cl-compiler@sail.stanford.edu
I've done a total rewrite on this issue. There are now three
proposals, which I think cover the range of possibilities. I think
that doing nothing with this issue would be the equivalent of
approving proposal NO.
I was going to send this out to X3J13 with our other proposals on
Monday. I realize that doesn't give you folks enough time to review
this properly, so I've marked it as being a draft. We can bring a
revised version to the meeting or decide not to bring it up for a
vote, if necessary.
Issue: PROCLAIM-ETC-IN-COMPILE-FILE
References: CLtL p. 182 [package functions],
p. 156 [PROCLAIM], p. 439 [COMPILE-FILE];
Issue COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS
Issue IN-PACKAGE-FUNCTIONALITY
Issue EVAL-WHEN-NON-TOP-LEVEL
Issue DEFINING-MACROS-NON-TOP-LEVEL
Category: CLARIFICATION, CHANGE, ADDITION
Edit History: 15 Sep 88, V1 by David Gray
23 Sep 88, V2 by Sandra Loosemore (summarize discussion)
11 Mar 89, V3 by Sandra Loosemore (rewrite)
Status: **DRAFT**
Problem Description:
Should the compiler treat top-level calls to PROCLAIM specially?
Page 182 of CLtL says that COMPILE-FILE needs to treat top-level calls
to the following package functions as though they were wrapped in an
(EVAL-WHEN (COMPILE LOAD EVAL) ...):
EXPORT IMPORT IN-PACKAGE MAKE-PACKAGE SHADOW
SHADOWING-IMPORT UNEXPORT UNUSE-PACKAGE USE-PACKAGE
CLtL is silent on whether top-level calls to PROCLAIM should also be
evaluated at compile-time, which presumably means they shouldn't be.
However, some implementations do evaluate PROCLAIM at compile-time.
In the model of how COMPILE-FILE works that is presented in issues
EVAL-WHEN-NON-TOP-LEVEL and DEFINING-MACROS-NON-TOP-LEVEL, the special
form EVAL-WHEN is the only thing that can cause compile-time evaluation
to occur. The compile-time side-effects of macros such as DEFMACRO
and DEFPACKAGE are explained by having them include EVAL-WHEN in their
expansions. Any functions that are treated specially, however, must
be included as special cases in the compiler.
Proposal IN-PACKAGE-FUNCTIONALITY:NEW-MACRO would remove the
requirement that the package functions be treated specially. Do we
wish to make an exception to the model for PROCLAIM?
Proposal PROCLAIM-ETC-IN-COMPILE-FILE:YES:
Require COMPILE-FILE to treat top-level calls to PROCLAIM as if they
were wrapped in an (EVAL-WHEN (COMPILE LOAD EVAL) ...).
Rationale:
Proclamations affect compilation semantics and should be made
available to the compiler.
Proposal PROCLAIM-ETC-IN-COMPILE-FILE:NO:
Clarify that calls to PROCLAIM should be treated the same as any
other function call. Users should wrap an explicit EVAL-WHEN around
top-level calls to PROCLAIM if they want them to affect compilation.
Rationale:
This makes the semantics of COMPILE-FILE more uniform and easier
to understand. In particular, if we remove the magic compile-time
behavior of the package functions, it seems silly to add another
exception for PROCLAIM.
Proposal PROCLAIM-ETC-IN-COMPILE-FILE:NEW-MACRO:
Add a new macro:
DEFPROCLAIM &rest decl-specs [Macro]
This macro PROCLAIMs the given <decl-specs>, which are not
evaluated. If a call to this macro appears at top-level in a file
being processed by the file compiler, the proclamations are also
made at compile-time. As with other defining macros, it is
unspecified whether or not the compile-time side-effects of a
DEFPROCLAIM persist after the file has been compiled.
Clarify that calls to PROCLAIM should be treated the same as any
other function call. Users should wrap an explicit EVAL-WHEN around
top-level calls to PROCLAIM if they want them to affect compilation,
or use the macro DEFPROCLAIM.
Rationale:
The macro makes the proclamations available to the compiler in such
a way that does not require any special exceptions to be made in
the model of how COMPILE-FILE works.
Current Practice:
The TI explorer apparently implements proposal YES, except that
(EVAL-WHEN (LOAD) (PROCLAIM '(OPTIMIZE ...))) doesn't do anything.
The Symbolics compiler has special top-level handling for PROCLAIM,
although the details are not clear.
Lucid does not evaluate calls to PROCLAIM at compile-time.
Cost to implementors:
Since implementations are already required to have a mechanism for
compile-time handling of the package functions, it would probably
only require minor adjustments to add handling for PROCLAIM.
Cost to users:
For proposal YES, users would have no way to suppress compile-time
evaluation of a top-level call to PROCLAIM. Wrapping it in an
(EVAL-WHEN (EVAL LOAD)...) wouldn't work under the model of how
EVAL-WHEN works in proposal EVAL-WHEN-NON-TOP-LEVEL:GENERALIZE-EVAL.
Under any of these proposals, some users would probably have to
make minor changes to their code.
Benefits:
Users will know what to expect when they use PROCLAIM.
Costs of Non-Adoption:
Users will not know what to expect when they use PROCLAIM.
Aesthetics:
At least two people consider requiring magic behavior for certain
top-level function calls to be "semantically bletcherous". Removing
all special cases for functions that are implicitly evaluated at
compile-time would simplify the model of how COMPILE-FILE works.
Programs look cleaner if EVAL-WHEN is only needed for unusual cases
instead of being required for the normal cases.
Discussion:
The first version of this writeup also included REQUIRE with PROCLAIM,
but we have now voted to remove REQUIRE from the language entirely.
It also specified that OPTIMIZE proclamations should only have a local
effect within the file being compiled. This was removed for
consistency with other compile-time side-effects (such as those from
DEFMACRO), where their persistence is explicitly left unspecified.
Loosemore favors proposal NO, but wouldn't oppose proposal NEW-MACRO.
-------
∂11-Mar-89 1304 CL-Compiler-mailer Issue: DEFINE-OPTIMIZER (Version 4)
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 11 Mar 89 13:01:47 PST
Received: from BOBOLINK.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 555145; Sat 11-Mar-89 15:58:46 EST
Date: Sat, 11 Mar 89 15:58 EST
From: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Issue: DEFINE-OPTIMIZER (Version 4)
To: cperdue@Sun.COM
cc: CL-Compiler@SAIL.Stanford.EDU
Message-ID: <890311155827.4.KMP@BOBOLINK.SCRC.Symbolics.COM>
[A revised proposal (v4) follows at the end of my reply to Cris.]
Date: Fri, 10 Mar 89 14:35:06 PST
From: cperdue@Sun.COM (Cris Perdue)
... I support Kent's proposal, with the following suggestions:
An additional function that corresponds to MACRO-FUNCTION for ordinary
macros. I include the proposed definition for this, which follows the
definition of MACRO-FUNCTION in CLtL, p144. This provides
primitive-level access, including the ability to undefine an
optimizer.
The current definition is agnostic about whether built-in system
optimizers are implemented using DEFINE-OPTIMIZER or vice versa.
This is important for two reasons:
- System optimizations may be type dispatched, as mentioned
in earlier discussions. In such cases, the general mechanism
proposed here could effectively define optimizations on type T
and not override more specific optimizations.
- Some systems may permit multiple optimizers for the same
function. In such a case, the optimizer set up by DEFINE-OPTIMIZER
may be only one of several.
Adding your OPTIMIZER-FUNCTION would expose the implementation, by
forcing us to say whether all the optimizers were captured, or only
the ones defined by DEFINE-OPTIMIZER. The interface I chose tries
deliberately to hide this issue in order to keep us from getting
bogged down in those other issues.
Similar arguments apply to the issue of undefining optimizers.
In both cases, I think you're describing very useful features and
I encourage you to (a) extend your implementation, (b) propose those
interfaces on the next standardization cycle.
The issue for now is that this proposal is essentially unchanged since
the last time I brought it up, but last round I made the mistake of
hinting that other issues might get involved so everyone immediately
went off in a million different directions and Sandra marked the
proposal as without clear support. I think she was right, and I think
what makes this time different is that the focus is just enough
different that we can all agree that this much functionality is ok.
Add to the problem description that sometimes an "operator" is defined
as a macro purely because its performance would be inadequate if
implemented as a function.
Good idea.
Also, I believe it would be best to state that in environments where
the compiler is prohibited from treating a function as INLINE,
optimizers are deactivated. In particular, the functions
OPTIMIZE-EXPRESSION and OPTIMIZE-EXPRESSION-1 do not transform their
expression argument if given an environment in which the function must
not be treated as INLINE. (I recognize that this is a bit of
a judgment call.)
The point here is to make user-defined optimizers follow the behavior
of optimizers built-in to compilers in their response to NOTINLINE.
I agree this is a problem, but I don't want to solve it this way because,
as you say, it's a judgment call, and I don't want to jeopardize the
whole thing if someone gets nervous. I will add wording saying that the
effects of having an OPTIMIZER on a function which is also proclaimed
INLINE are ``unspecified but harmless'' (in that both optimizations and
inlining are intended to be semantics-preserving operations).
-----
Forum: Compiler
Issue: DEFINE-OPTIMIZER
References: Issue SYNTACTIC-ENVIRONMENT-ACCESS
Category: ADDITION
Edit history: 28-Sep-88, Version 1 by Pitman
10-Mar-89, Version 2 by Pitman (clarifications, new example),
10-Mar-89, Version 3 by Pitman & Loosemore
Status: Ready for release
Problem Description:
Often a general functional interface could be bypassed given explicit
knowledge of the arguments passed. This may happen when the arguments
are constant (or otherwise inferrable), an argument type is known (eg,
due to use of THE or DECLARE), or when some particular pattern of
optional, rest or keyword arguments is apparent.
Most implementations provide internally for optimization of generalized
function call interfaces to more specialized ones, but such an
optimization facility is not provided to Common Lisp users.
The absence of this facility in a portable fashion means that some
CL programs run slower than they need to in some implementations, or
else that some operators that should be implemented as functions end
up getting implemented as macros to assure needed efficiency.
Proposal (DEFINE-OPTIMIZER:NEW-FACILITY):
Introduce a facility for declaring compiler optimizations.
DEFINE-OPTIMIZER name arglist {declaration}* {form}* [Macro]
Defines a compiler optimizer for a function named NAME. The ARGLIST,
DECLARATIONS, and FORMS are treated exactly like the arglist,
declarations, and forms in a DEFMACRO. (The arglist may include
&ENVIRONMENT and &WHOLE.)
The argument NAME must name a function which has been previously
defined. The effects of defining an optimizer for a locally or
globally defined macro, a locally defined function, or a special
form are undefined.
When the optimizer is invoked, the forms are executed in the context
of bindings specified by the arglist, and two values are yielded,
RESULT and VALID-P. (If either of the first or second return value
is non-NIL, then the first return value is considered valid).
If the result is valid, it is a form which is preferrable to evaluate
instead of the indicated call.
If a call to DEFINE-OPTIMIZER appears at top-level in a file
being processed by the file compiler, it also makes the optimizer
known at compile-time (similar to the way DEFMACRO makes a macro
definition known to the compiler).
OPTIMIZE-EXPRESSION-1 form env [Function]
Similar to MACROEXPAND-1. Invokes the optimizers for the top level of
FORM, but does not iterate on the result. Returns two values:
RESULT and CHANGED-P.
Note: If an optimizer returns a result which is not valid,
OPTIMIZE-EXPRESSION-1 hides the fact by returning FORM,NIL
rather than NIL,NIL.
OPTIMIZE-EXPRESSION form env [Function]
Iterates calling OPTIMIZE-EXPRESSION-1 until the CHANGED-P result
is NIL.
An implementation must save optimizer definitions created by
DEFINE-OPTIMIZER in case OPTIMIZE-EXPRESSION is attempted, but is
not actually required to call OPTIMIZE-EXPRESSION itself. Interpreters,
for example, may choose to just call the unoptimized form.
Using FLET and MACROLET shadow not only functions and their SETF methods,
but also their optimizers. No portable facility is provided for creating
locally defined optimizers.
The effect of defining optimizations for functions on the LISP package
is not defined. (In some implementations, this would clobber or conflict
with existing advice that may be of higher quality.)
The editor is advised that a non-binding style note such as the
following would also be appropriate:
In general, it is poor style for a programmer to define optimizers for
functions that he does not maintain. This is because the correct
implementation of an optimizer for a function usually depends on an
understanding of the internals of that function. As such, a function
definition and any optimizers should be maintained as a unit so that
they can changes in either can be synchronized as appropriate with the
other.
The effect of using DEFINE-OPTIMIZER on a function declared to be
INLINE is ``unspecified but harmless'' (per new Error Terminology).
That is, since both operations (optimization and inlining) are intended
to be semantics-preserving, no functional difference should be observed.
However, in some implementations the presence of an optimizer may thwart
the ability to inline, or vice versa. Writers of portable are encouraged
to use either DEFINE-OPTIMIZER or (PROCLAIM '(INLINE ...)) but not both.
Example:
;; These examples are taken literally from the Macsyma sources,
;; modified only to change DEFOPT to DEFINE-OPTIMIZER. The comments
;; were specially written for the X3J13 audience.
;; M+ is adds a Macsyma expression to another Macsyma expression.
;; The Macsyma internal representation for the sum of X and Y is
;; ((MPLUS) X Y). A all the real work is done by SIMPLIFY, which
;; reduces the expression as needed necessary. However, SIMPLIFY
;; is very complicated, and considerable speed can be gained by
;; entering it at specific known places.
(DEFUN M+ (&REST TERMS)
(PROTECT-&REST-VARIABLE TERMS)
(SIMPLIFY `((MPLUS) ,@TERMS)))
(DEFINE-OPTIMIZER M+ (&REST TERMS)
(COND ((= (LENGTH TERMS) 2) `(ADD2* ,@TERMS))
(T `(ADDN (LIST ,@TERMS) NIL))))
;; M- negates a Macsyma expression, or substracts two Macsyma
;; expressions. Once you figure out which of the two operations is
;; to be done, the problem is similar to that of M+ above. However,
;; often the decision can be made at compile time. In this case,
;; INLINE functions would have worked ok, except that not all
;; implementations do inlining, and even those that do may fail to
;; recognize that EXP2 being NIL means that a test can be eliminated
;; or dead code can be eliminated. Using optimizers is far more
;; likely to be useful in practice.
(DEFUN M- (EXP1 &OPTIONAL (EXP2 NIL EXP2P))
(IF (NOT EXP2P)
(M--INTERNAL-NEGATE EXP1)
(M--INTERNAL-SUBTRACT EXP1 EXP2)))
(DEFINE-OPTIMIZER M- (EXP1 &OPTIONAL (EXP2 NIL EXP2P))
(IF (NOT EXP2P)
`(M--INTERNAL-NEGATE ,EXP1)
`(M--INTERNAL-SUBTRACT ,EXP1 ,EXP2)))
Rationale:
Many large portable applications expect such a facility on an
implementation-specific basis. Others would use one if available.
Even if implementations don't use the provided optimizers primitively,
user macros and code-walkers can invoke them, so the facility wouldn't
be completely useless even in those implementations.
Current Practice:
Symbolics Genera provides an optimizer facility which is more elaborate
but not fundamentally incompatible with this facility.
Many (if not most) serious implementations provide a similar facility.
For example, Lucid provides "compiler macros" which serve the same
purpose.
Cost to Implementors:
Since the implementation is not required to use this facility, the
cost of providing the proposed support is very small.
Cost to Users:
None. This change is upward compatible.
Cost of Non-Adoption:
Portable code would be slower than necessary in some situations.
Benefits:
Some existing non-portable code could become portable.
Aesthetics:
Providing a separate optimizer definition from a main function definition
makes a possibility that the optimizer and main function could drift out
of synch. However, most places where this gets used in the first place
are places where speed is of paramount importance and the programmer is
willing to invest effort in maintaining things correctly and to accept the
risk of lossage if s/he fails.
This is a fairly clean and simple extension which adds significant
power to the compiler.
Discussion:
Pitman strongly supports this proposal, the design of which is modeled
directly after that which has been used in Macsyma for many years.
Information about argument type can come from two different sources:
THE and declarations (via PROCLAIM or DECLARE). The former information
is portably accessible, the latter is not. While a separate proposal
(SYNTACTIC-ENVIRONMENT-ACCESS) for allowing program access to type
declarations would be make this facility more useful, it is still
quite useful without it, as the examples from Macsyma illustrate.
Some implementations provide a way to provide more than one optimizer
for the same function. A multiple optimizer facility can be written
in terms of this simpler facility and vice versa, so the simpler of
the two facilities is proposed here.
Some people have suggested that they would like to see a pattern
matching facility integrated into this facility. The design of a
facility that would satisfy everyone would take a lot of time and
effort. At this point, there is no chance that the design of such a
facility would occur in time for acceptance into the standard.
The choice is this or nothing. Pitman thinks the language is much
better off with some form of optimization support than none.
Loosemore says:
Although I don't really think this is an essential feature to include
in the standard, I don't have any strong objection to adding it. If
people think it's a good idea to provide a standard interface for this
kind of thing, this is a good proposal for doing it -- it's fairly
simple, doesn't introduce any radically new ideas, and is general
enough to allow alternate interfaces (such as the pattern matcher) to
be layered on top of it.
∂11-Mar-89 1321 CL-Compiler-mailer Re: issue MACRO-ENVIRONMENT-EXTENT, version 2
Received: from ti.com by SAIL.Stanford.EDU with TCP; 11 Mar 89 13:20:53 PST
Received: by ti.com id AA10598; Sat, 11 Mar 89 15:19:03 CST
Received: from Kelvin by tilde id AA15498; Sat, 11 Mar 89 15:09:15 CST
Message-Id: <2814642505-9565798@Kelvin>
Sender: GRAY@Kelvin.csc.ti.com
Date: Sat, 11 Mar 89 15:08:25 CST
From: David N Gray <Gray@DSG.csc.ti.com>
To: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Cc: cl-compiler@sail.stanford.edu,
"David A. Moon" <Moon@STONY-BROOK.SCRC.Symbolics.COM>,
"Kim A. Barrett" <IIM%ECLA@ECLC.USC.EDU>
Subject: Re: issue MACRO-ENVIRONMENT-EXTENT, version 2
In-Reply-To: Msg of Sat, 11 Mar 89 09:03:42 MST from sandra%defun@cs.utah.edu (Sandra J Loosemore)
> This is too vague. What does "scope of that information" mean? Dynamic
> scope? Lexical scope? You need to say what the *extent* is, not the
> scope.
I agree the wording could use improvement. The idea is that the extent
of the information is the time during which the processor is processing
the code within that scope.
> An example may clarify my confusion. :-) Suppose I'm using
> LOCAL-TYPE-DECLARE as implemented in the writeup (but with
> COPY-ENVIRONMENT added to it, of course), and then I say something
> like:
>
> (defun f (x)
> (local-type-declare ((x fixnum))
> #'(lambda (y)
> (local-type-declare ((y fixnum))
> (+ (typed var x) y)))
> ))
>
> Is this legitimate? Let's assume that the interpreter doesn't do a
> code walk. The closure returned when you call F has indefinite
> extent, yet it contains a reference to the environment established by
> the lexically enclosing LOCAL-TYPE-DECLARE. So if this is legal,
> the environment has to have indefinite extent too.
This wouldn't be any problem when compiled, but you are right that an
interpreter that expands macros during execution would need to use a
macro expansion environment with indefinite extent. I hadn't thought of
that difference before. Of course, though, such an implementation would
have to support indefinite-extent macro environments internally in order
to correctly implement closures regardless of the
MACRO-ENVIRONMENT-EXTENT issue, so there shouldn't be any hardship for
it to permit more explicit use of saved environments. In any case, I
have no objection to saying that local macro definitions have indefinite
extent, it's just the other things that environments might be used for
that worries me.
> Rather than trying to add all these ill-defined qualifications and
> restrictions, if we absolutely cannot agree to give environments
> indefinite extent (or to provide a copier to give them indefinite
> extent), I would much rather see a simple statement that you can't
> assume more than dynamic extent.
Maybe a simpler way to look at it is that you can't expect to be able to
extract from an environment object information that would not be present
(although possibly shadowed) in the environment that would be given to a
macro expander in the current lexical context. Thus, your example would
be permitted because the saved environment used by the local macro
TYPED-VAR is a lexical ancestor of the environment that the expander is
given directly. You aren't trying to access a definition that doesn't
exist anymore, just something that might be shadowed in the current
environment.
Similarly, a saved environment could be used to access class definitions
later during COMPILE-FILE, because all environments after that time will
include the same class definitions.
Does that make the concept any clearer?
> > Within COMPILE-FILE, class definitions resulting from a top-level
> > DEFCLASS can not be accessed after processing of the file is complete.
>
> As far as I've been able to tell, there is nothing in CLOS chapters 1
> & 2 that says that DEFCLASS *must* store class definitions in any
> particular way that involves environment objects.
The problem is with FIND-CLASS, which is documented on page 2-48 to
accept an environment argument:
"The optional environment argument is the same as the &ENVIRONMENT
argument to macro expansion functions. It is typically used to
distinguish between compile-time and run-time environments."
Perhaps this feature ought to be moved to chapter 3, since it doesn't
help much to have this included in the standard without the supporting
specifications that makes it meaningful.
∂11-Mar-89 1502 CL-Compiler-mailer Re: issue CLOS-MACRO-COMPILATION, version 1
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by SAIL.Stanford.EDU with TCP; 11 Mar 89 15:02:28 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 555181; Sat 11-Mar-89 17:58:31 EST
Date: Sat, 11 Mar 89 17:58 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Re: issue CLOS-MACRO-COMPILATION, version 1
To: David N Gray <Gray@DSG.csc.ti.com>
cc: Sandra J Loosemore <sandra%defun@cs.utah.edu>, cl-compiler@sail.stanford.edu,
common-lisp-object-system@sail.stanford.edu
In-Reply-To: <2814566813-5018117@Kelvin>
Message-ID: <19890311225826.2.MOON@EUPHRATES.SCRC.Symbolics.COM>
Date: Fri, 10 Mar 89 18:06:53 CST
From: David N Gray <Gray@DSG.csc.ti.com>
> Proposal CLOS-MACRO-COMPILATION:MINIMAL:
...
> DEFMETHOD:
>
> * The method is not callable at compile-time. If there is a generic
> function with the same name at compile-time, compiling a DEFMETHOD
> will not add the method to that generic function.
>
> [This also seems to imply that tests for existence of the generic
> function, lambda-list congruence, etc. must not happen until
> load time.]
No, an implementation should be permitted to check for lambda-list
congruence between methods defined in the same file; this doesn't
require any reference to the resident generic function definition. If
the file doesn't include a DEFGENERIC, then the first DEFMETHOD defines
the compile-time generic function attributes, and subsequent methods can
be checked against that.
Agreed. I would phrase it differently: all the DEFGENERICs and DEFMETHODS
for a given generic function name in a given compilation unit can be checked
against each other for lambda-list congruence.
> DEFINE-METHOD-COMBINATION:
>
> * The method combination can be used in a subsequent DEFGENERIC. If it
> is referenced, the body of a long form of method combination must be
> evaluable at compile-time.
But if methods are not installed at compile time and generic functions
are not callable at compile time, then I don't think there is any
situation in which the method combination body could be executed at
compile-time.
Some implementations compose and compile effective methods at compile
time, which of course requires evaluating the body of the
define-method-combination at compile time.
I haven't read Sandra's proposal yet.
∂11-Mar-89 1531 CL-Compiler-mailer Re: issue MACRO-ENVIRONMENT-EXTENT, version 2
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 11 Mar 89 15:31:22 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA10960; Sat, 11 Mar 89 16:29:10 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA00784; Sat, 11 Mar 89 16:29:08 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903112329.AA00784@defun.utah.edu>
Date: Sat, 11 Mar 89 16:29:06 MST
Subject: Re: issue MACRO-ENVIRONMENT-EXTENT, version 2
To: David N Gray <Gray@DSG.csc.ti.com>
Cc: sandra%defun@cs.utah.edu (Sandra J Loosemore),
cl-compiler@sail.stanford.edu,
"David A. Moon" <Moon@STONY-BROOK.SCRC.Symbolics.COM>,
"Kim A. Barrett" <IIM%ECLA@ECLC.USC.EDU>
In-Reply-To: David N Gray <Gray@DSG.csc.ti.com>, Sat, 11 Mar 89 15:08:25 CST
Hmmm. It sounds to me like you want the environment to have an extent
similar to the extent of the special bindings made by COMPILER-LET.
How about some wording like:
The extent of the environment objects passed to MACROEXPAND or
MACROEXPAND-1 by COMPILE-FILE, COMPILE, and EVAL is the dynamic extent
during which the macro and its expansion are processed. In
particular, the extent includes the expansion of any other macro calls
appearing lexically within the form returned from MACROEXPAND or
MACROEXPAND-1.
I think this would fit in with issue SYNTACTIC-ENVIRONMENT-ACCESS, but
I'm still confused about why you would still need COPY-ENVIRONMENT.
One other thing that's still not really clear to me is why you think
this is better than just saying they have (or can be copied to have)
indefinite extent. It would make the typed-variable example work but
not be of much use for macro-caching. Can you provide a more specific
rationale than just saying that it gives implementors more flexibility?
-Sandra
-------
∂11-Mar-89 1621 CL-Compiler-mailer Issue CONSTANT-COMPILABLE-TYPES, version 7
Received: from lucid.com by SAIL.Stanford.EDU with TCP; 11 Mar 89 16:21:33 PST
Received: from challenger ([192.9.200.17]) by heavens-gate.lucid.com id AA01042g; Sat, 11 Mar 89 16:14:32 PST
Received: by challenger id AA05885g; Sat, 11 Mar 89 16:09:56 PST
Date: Sat, 11 Mar 89 16:09:56 PST
From: Richard P. Gabriel <rpg@lucid.com>
Message-Id: <8903120009.AA05885@challenger>
To: cl-compiler@sail.stanford.edu
Subject: Issue CONSTANT-COMPILABLE-TYPES, version 7
I guess I pretty strongly object to leaving functions out of the list
of constants that can appear in compiled code. The part that's
disturbing is that such non-Lispy things like arrays, hashtables, and
pathnames get better treatment than functions, the most Lispy part of
Common Lisp. I wonder how many implementations will be forced to come
within an inch of the required functionality to implement a first-rate
CLOS?
The specification of the subset of functions that are acceptable as
compiled constants cannot be tested for within Common Lisp itself.
I suggest we ask implementors (Lucid included) to bite the bullet and
handle this case correctly. Won't our grandchildren appreciate us
treating Common Lisp like Lisp and not like PASCAL?
Also, as will be seen in a subsequent message, there might be other
compiler issues that can be treated my esthetically were this issue
fixed up.
-rpg-
∂11-Mar-89 1643 CL-Compiler-mailer Issue CLOS-MACRO-COMPILATION, version 1
Received: from lucid.com by SAIL.Stanford.EDU with TCP; 11 Mar 89 16:43:21 PST
Received: from challenger ([192.9.200.17]) by heavens-gate.lucid.com id AA01057g; Sat, 11 Mar 89 16:36:21 PST
Received: by challenger id AA05917g; Sat, 11 Mar 89 16:31:46 PST
Date: Sat, 11 Mar 89 16:31:46 PST
From: Richard P. Gabriel <rpg@lucid.com>
Message-Id: <8903120031.AA05917@challenger>
To: cl-compiler@sail.stanford.edu
Subject: Issue CLOS-MACRO-COMPILATION, version 1
Sandra writes:
``The description of DEFMETHOD in CLOS chapter 2 talks about calling
FBOUNDP and signalling an error if the function is not a generic
function, or if it is a generic function but the lambda list of the
method is not congruent. Clearly this shouldn't happen at
compile-time. I agree that the behavior you suggest makes more sense.
''
To be slightly pissant about it, Chapter 2 doesn't say this exactly.
It says that if FBOUNDP of the function specifier is NIL, certain
actions take place, and if other conditions hold, errors are
signalled. The wording is chosen so as to not imply that FBOUNDP is
called at any particular time, nor is it necessary for FBOUNDP to ever
be called. Just as in the current ADJUST-ARRAY issue, the operative
phrase is about whether the result of a call would be something
particular.
Chapter 1 and 2 are written in such a way that the semantics of a
correct, well-formed program are specified. The model that Linda and I
(the two writers) had in mind was that of well-formed programs being
either loaded into a fresh Lisp or of being compiled in a fresh Lisp
and then loaded into a fresh Lisp. We tried to specify the minimally
required semantics of such well-formed programs.
The question I have about the process going on with respect to the
CLOS-MACRO-COMPILATION issue is whether the fine-grained behavior of
CLOS under various compilation/evaluation situations is being
over-specified.
At this stage of the game I worry that we might go a little too far in
one direction in specification when we are actually engaged in design
work. This isn't intended to be a criticism of any committees, but I
would feel a lot more comfortable with a conservative specification
that defined well-formed programs being loaded or compiled in fresh
Common Lisps with a pretty simple eval-when model that is easier to
specify and which makes it easy for all but the hairiest
compilation-environment-frobbing programs to be written. What tips me
off to this is the heroic effort I see Sandra and the other compiler
committee members making to understand the ultimately hairy
situations.
I know this makes the specification of Common Lisp more like
specifications of Ada or PASCAL, but at least we have a better chance
of not blowing it.
-rpg-
∂11-Mar-89 1658 Common-Lisp-Object-System-mailer Issue: LOAD-OBJECTS (Version 3)
Received: from lucid.com by SAIL.Stanford.EDU with TCP; 11 Mar 89 16:58:32 PST
Received: from challenger ([192.9.200.17]) by heavens-gate.lucid.com id AA01066g; Sat, 11 Mar 89 16:51:27 PST
Received: by challenger id AA05936g; Sat, 11 Mar 89 16:46:51 PST
Date: Sat, 11 Mar 89 16:46:51 PST
From: Richard P. Gabriel <rpg@lucid.com>
Message-Id: <8903120046.AA05936@challenger>
To: CL-Cleanup@sail.stanford.edu, CL-Compiler@sail.stanford.edu,
Common-Lisp-Object-System@sail.stanford.edu
Subject: Issue: LOAD-OBJECTS (Version 3)
I was a little surprised to see that this proposal talks about load
forms instead of load functions (which goes to show how much I've been
paying attention). After some thought and consultation with Moon, I
realized that part of it was that compiled functions could not be
compiled constants. If we were to allow such constants, I would propose
we consider the alternative of load functions.
The model would be that when objects are being either prepared for
dumping or are being dumped, at certain points the generic function
MAKE-LOAD-FUNCTION would be invoked on objects that needed to be
re-created later. It would return either one or two functions. The
first is a function of 0 arguments that does the initial creation, and
the second is (if present) a function of 1 argument, which is the
initializer. If present it is applied to the created instance. This
simplifies the naming problem in the current proposal, which, while
clever, is a little unpalatable. In particular, it introduces yet
another way to think about variables.
I think people will find the macro approach (the current approach)
baroque, partly because the approach is best understood by thinking of
an input phase to a compiler or some such program, rather than by
thinking about an output phase when everything has already been supposedly
created. For example, when I read the current proposal, I imagined it
in the FASDUMP phase.
One drawback of my proposal is that the function approach is a little
more verbose in some cases. I also think it is subject to more
circularity errors by novices than the macro approach. On the other
hand, the functional approach makes one think about the issues a
little harder when writing the code, which is possibly a good thing.
Here are the examples in the macro proposal:
;; Example 1
(defclass my-class ()
((a :initarg :a :reader my-a)
(b :initarg :b :reader my-b)
(c :accessor my-c)))
(defmethod shared-initialize ((self my-class) ignore &rest ignore)
(unless (slot-boundp self 'c)
(setf (my-c self) (some-computation (my-a self) (my-b self)))))
(defmethod make-load-function ((self my-class))
(let ((name (class-name (class-of self)))
(a (my-a self))
(b (my-b self)))
#'(lambda () (make-instance name :a a :b b))))
Here the computations of NAME, A, and B must be outside the function
#'(lambda ...) so that they get evaluated in the right environment to
avoid a circular (self-referential) structure. For this to work, the
faslout of #'(lambda ...) must also notice any constants or such
things that need similar treatment, which will get NAME, A, and B, if
needed.
;; Example 2
(defclass my-frob ()
((name :initarg :name :reader my-name)))
(defmethod make-load-function ((self my-frob))
(let ((name (my-name self)))
#'(lambda () (find-my-frob name :if-does-not-exist :create))))
Maybe NAME is not something to worry about, but SELF cannot appear in
the #'(lambda ...).
;; Example 3 (expanded to do a hairy thing that cannot be easily done
;; in the macro approach).
(defclass tree-with-parent () ((parent :accessor tree-parent)
(curious-facts :accessor tree-foma)
(children :initarg :children)))
(defmethod make-load-function ((x tree-with-parent))
(let ((class (class-of x))
(children (slot-value x 'children))
(random-info-at-dump-time (compute-random-info x))
(more-random-info-at-creation-time ())
(parent (slot-value x 'parent)))
(flet ((initialize (x)
(setf (tree-foma x)
(combine-info random-info-at-dump-time
random-info-at-creation-time))
(setf (tree-parent x) parent)))
(values
;; creation
#`(lambda ()
(setq more-random-info-at-creation-time
(compute-more-random-info))
(make-instance class :children children))
;; initialization
#'initialize))))
One can imagine the shared lexical environment of the creator and initializer
being a high-bandwidth channel for information, such as the important
information passed in the above example.
Finally, I wouldn't use the name MAKE-LOAD-FUNCTION-USING-SLOTS,
because the structure of the name ...-USING-SLOTS is like
...-USING-CLASS, which is named that way to inform the programmer that
he can discriminate on the metaclass. Maybe,
MAKE-LOAD-FUNCTION-PRESERVING-SLOTS?
-rpg-
∂11-Mar-89 1718 CL-Compiler-mailer Re: Issue CONSTANT-COMPILABLE-TYPES, version 7
Received: from Sun.COM by SAIL.Stanford.EDU with TCP; 11 Mar 89 17:18:10 PST
Received: from snail.Sun.COM by Sun.COM (4.1/SMI-4.0)
id AA19653; Sat, 11 Mar 89 17:18:56 PST
Received: from clam.sun.com by snail.Sun.COM (4.1/SMI-4.0)
id AA20560; Sat, 11 Mar 89 17:15:13 PST
Received: by clam.sun.com (4.0/SMI-4.0)
id AA18680; Sat, 11 Mar 89 17:18:44 PST
Date: Sat, 11 Mar 89 17:18:44 PST
From: cperdue@Sun.COM (Cris Perdue)
Message-Id: <8903120118.AA18680@clam.sun.com>
To: cl-compiler@sail.stanford.edu, rpg@lucid.com
Subject: Re: Issue CONSTANT-COMPILABLE-TYPES, version 7
Dick,
> I guess I pretty strongly object to leaving functions out of the list
> of constants that can appear in compiled code. The part that's
> disturbing is that such non-Lispy things like arrays, hashtables, and
> pathnames get better treatment than functions, the most Lispy part of
> Common Lisp. I wonder how many implementations will be forced to come
> within an inch of the required functionality to implement a first-rate
> CLOS?
Thanks for the note. I feel a good deal the way you do about the
importance of functions.
> The specification of the subset of functions that are acceptable as
> compiled constants cannot be tested for within Common Lisp itself.
Sigh. Did we remove compiled-function-p from the language? My
thought was that functions not closing over any variables and
for which this test would return NIL would have to be accepted
as compiled constants.
> I suggest we ask implementors (Lucid included) to bite the bullet and
> handle this case correctly. Won't our grandchildren appreciate us
> treating Common Lisp like Lisp and not like PASCAL?
There are technical issues regarding closures, and since you are
interested, I solicit your opinion:
There is of course a set of lexical variables and their state
associated with each closure. Most objects resulting from
file-compilation and loading may be made immutable (as specified
in the proposal). This would seem inappropriate for the closed-over
variables and the objects referred to by those variables.
Is this your view also?
There is also an issue concerning how carefully to preserve the
identities of variables closed over by more than one compiled and
reloaded closure. Finally (I hope this is all) there are issues
concerning how carefully to preserve the identities ("EQ-ness") of
objects referred to by closed-over variables. Uninterned symbols
presented similar issues, and careful preservation of identity turned
out to be desired there.
Basically I have been reluctant to take on the picky technical
issues here, but I hope someone else will have more energy on this
than I have. A cleanup proposal that would amend/augment
CONSTANT-COMPILABLE-TYPES:SPECIFY would be welcome.
-Cris
∂11-Mar-89 1745 Common-Lisp-Object-System-mailer Re: Issue: LOAD-OBJECTS (Version 3)
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 11 Mar 89 17:45:12 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA13023; Sat, 11 Mar 89 18:43:00 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA00878; Sat, 11 Mar 89 18:42:57 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903120142.AA00878@defun.utah.edu>
Date: Sat, 11 Mar 89 18:42:56 MST
Subject: Re: Issue: LOAD-OBJECTS (Version 3)
To: Richard P. Gabriel <rpg@lucid.com>
Cc: CL-Cleanup@sail.stanford.edu, CL-Compiler@sail.stanford.edu,
Common-Lisp-Object-System@sail.stanford.edu
In-Reply-To: Richard P. Gabriel <rpg@lucid.com>, Sat, 11 Mar 89 16:46:51 PST
I haven't been paying too much attention to this issue either -- I've
been trusting Moon to do the right thing on the assumption that he
knew more about it than I did. I think his latest proposal does look
reasonable. However, if there is disagreement about it, I might as
well suggest yet another alternative:
Have two generic functions, not one. The first would get called by
compile-file and it would return a list of components (or whatever)
that are required to reconstruct the object. The compiler would dump
this list of objects in its usual way. The loader would apply the
second generic function to this list to reconstruct the object. It
avoids the nasty syntax you object to, doesn't require functions to be
dumpable, doesn't require any special support for circular constants,
and ought to be real easy to add to the compiler/loader. (You could
essentially convert the constant into a LOAD-TIME-VALUE expression.)
-Sandra
-------
∂11-Mar-89 1806 CL-Compiler-mailer Re: Issue CONSTANT-COMPILABLE-TYPES, version 7
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 11 Mar 89 18:05:57 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA13405; Sat, 11 Mar 89 19:03:50 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA00900; Sat, 11 Mar 89 19:03:48 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903120203.AA00900@defun.utah.edu>
Date: Sat, 11 Mar 89 19:03:47 MST
Subject: Re: Issue CONSTANT-COMPILABLE-TYPES, version 7
To: cperdue@Sun.COM (Cris Perdue)
Cc: cl-compiler@sail.stanford.edu, rpg@lucid.com
In-Reply-To: cperdue@Sun.COM (Cris Perdue), Sat, 11 Mar 89 17:18:44 PST
The problem with not having a predicate to test functions that meet
the requirements is that there's no predicate for testing whether a
function is closed over any variables. See CLtL p. 89.
A further technical problem with dumping functions is that for
Lisp-callable functions that are written in microcode, machine
language, C, or whatever, it's hard to define how they should be
dumped. This applies not only to functions that the user loads with a
foreign-function interface, but to built-in functions provided as part
of the Lisp kernel. For example, suppose the kernel provides some
Lisp-callable C functions to interface to the file system, math
library, or whatever. These might be represented just as a stub that
jumps off to the address that the C linker/loader just happened to
relocate the C function at (which may differ from one invocation of
the Lisp image to the next). There's no way to determine the size of
a C function from within C, and even if you did you may not be able to
dump and reload it because maybe the C compiler didn't generate
relocatable code.
I suppose one alternative would be to say that such functions are not
FUNCTIONP but to extend APPLY, FUNCALL, etc. to accept them. Is it
legal for functions defined in the standard not to be FUNCTIONP?
-Sandra
-------
∂12-Mar-89 1002 CL-Compiler-mailer issue SYNTACTIC-ENVIRONMENT-ACCESS, version 4
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 12 Mar 89 10:02:16 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA25589; Sun, 12 Mar 89 11:00:06 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA01408; Sun, 12 Mar 89 11:00:02 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903121800.AA01408@defun.utah.edu>
Date: Sun, 12 Mar 89 10:59:59 MST
Subject: issue SYNTACTIC-ENVIRONMENT-ACCESS, version 4
To: cl-compiler@sail.stanford.edu
I've done a rather substantial re-organization on this issue. There
are now three proposals, SMALL, MEDIUM, and LARGE. I did this because
I personally didn't feel comfortable with standardizing more than the
material in proposal SMALL at this point, and maybe others will also
be more inclined to vote for a minimal proposal than a larger one that
includes material of less obvious usefulness.
I've made the change to AUGMENT-ENVIRONMENT (adding a :DECLARE argument)
that I suggested in a previous message.
I've also removed the RECORD-ENVIRONMENT-CLEANUP function. I couldn't
think of any good examples of how or why one would want to use this,
that couldn't also be achieved by putting an explicit UNWIND-PROTECT
in the body of the WITH-REMOTE-ENVIRONMENT.
Forum: Compiler
Issue: SYNTACTIC-ENVIRONMENT-ACCESS
References: CLtL Chapter 8: Macros,
Issue MACRO-FUNCTION-ENVIRONMENT,
Issue GET-SETF-METHOD-ENVIRONMENT,
Issue COMPILE-FILE-ENVIRONMENT
Related Issues: Issue FUNCTION-NAME,
Issue PROCLAIM-LEXICAL
Issue MACRO-ENVIRONMENT-EXTENT
Category: ADDITION
Edit history: Version 1, 2-Oct-88, Eric Benson
Version 2, 17-Feb-89, Kim A. Barrett
Version 3, 9-Mar-89, Kim A. Barrett (respond to comments)
Version 4, 12-Mar-89, Sandra Loosemore (more revisions)
Status: **DRAFT**
Problem description:
When macro forms are expanded, the expansion function is called with
two arguments: the form to be expanded, and the environment in which
the form was found. The environment argument is of limited utility.
The only use sanctioned currently is as an argument to MACROEXPAND or
MACROEXPAND-1 or passed directly as an argument to another macro
expansion function. Recent cleanup issues propose to allow it as an
argument to MACRO-FUNCTION and to GET-SETF-METHOD.
It is very difficult to write a code walker that can correctly handle
local macro and function definitions, due to insufficient access to
the information contained in environments and the inability to
augment environments with local definitions.
Some people believe that the CLOS meta-object protocol will require the
ability to distinguish between remote environments used for compiling
to a file, from local environments used for processing by EVAL or
COMPILE. (However, there is no requirement in chapters 1 & 2 of the
CLOS spec that things be done this way.)
There are three proposals; SMALL, MEDIUM, and LARGE.
Proposal (SYNTACTIC-ENVIRONMENT-ACCESS:SMALL):
The following functions provide information about syntactic
environment objects. In all of these functions the argument named ENV
is an environment of the sort received by the &ENVIRONMENT argument to
a macro or as the environment argument for EVALHOOK. Optional "env"
arguments default to NIL, which respresents the local null lexical
environment (containing only global definitions and proclamations
that are present in the runtime environment). All of these functions
should signal an error of type TYPE-ERROR if the value of an
environment argument is not a syntactic environment.
VARIABLE-KIND variable &optional env [Function]
VARIABLE is a symbol. This function returns one of the following
symbols, depending on the type of definition or binding which is
apparent in ENV.
NIL There is no apparent definition or binding for variable.
:SPECIAL VARIABLE refers to a special variable, either declared
or proclaimed.
:LEXICAL VARIABLE refers to a lexical variable.
:SYMBOL-MACRO VARIABLE refers to a SYMBOL-MACROLET binding.
:CONSTANT VARIABLE refers to a named constant, defined by
DEFCONSTANT.
[Note: If issue PROCLAIM-LEXICAL passes, then the :LEXICAL result
will also refer to variables proclaimed lexical.]
Example:
(DEFMACRO KIND-OF-VARIABLE (VAR &ENVIRONMENT ENV)
`',(VARIABLE-KIND VAR ENV))
(DEFVAR A)
(DEFUN TEST ()
(LET (B)
(LET (C)
(DECLARE (SPECIAL C))
(SYMBOL-MACROLET ((D ANYTHING))
(LIST (KIND-OF-VARIABLE A)
(KIND-OF-VARIABLE B)
(KIND-OF-VARIABLE C)
(KIND-OF-VARIABLE D)
(KIND-OF-VARIABLE E))))))
(TEST) -> (:SPECIAL :LEXICAL :SPECIAL :SYMBOL-MACRO NIL)
FUNCTION-KIND function &optional env [Function]
FUNCTION is a function name. This function returns two values,
depending on the type of function definition or function binding
which is apparent for FUNCTION in ENV.
NIL There is no apparent definition for FUNCTION.
:FUNCTION FUNCTION refers to a function.
:MACRO FUNCTION refers to a macro.
:SPECIAL-FORM FUNCTION refers to a special form.
The second value specifies whether the definition is local or
global. If local, the second value is true, and it is false when
the definition is global.
Some function names may refer to both a global macro and a global
special form. In such a case, the macro takes precedence, and
:MACRO is returned as the first value.
[Note: The use of "function name" rather than "symbol" as the
description of the function argument is intended to be compatible
with the various proposals to extend the syntax of function
specifiers. If no such change actually occurs then this would only
refer to symbols.]
Example:
(DEFMACRO KIND-OF-FUNCTION (FUNCTION-NAME &ENVIRONMENT ENV)
`',(FUNCTION-KIND FUNCTION-NAME ENV))
(DEFUN A ())
(DEFMACRO B ())
(DEFUN TEST ()
(FLET ((C ()))
(MACROLET ((D ()))
(MULTIPLE-VALUE-CALL #'LIST
(KIND-OF-FUNCTION A)
(KIND-OF-FUNCTION B)
(KIND-OF-FUNCTION QUOTE)
(KIND-OF-FUNCTION C)
(KIND-OF-FUNCTION D)
(KIND-OF-FUNCTION E)))))
(TEST) -> (:FUNCTION NIL
:MACRO NIL
:SPECIAL-FORM NIL
:FUNCTION T
:MACRO T
NIL NIL)
VARIABLE-TYPE variable &optional env [Function]
VARIABLE is a symbol. This function returns the type specifier
associated with the variable named by the symbol in the environment.
If no explicit association exists, either by PROCLAIM or DECLARE,
then the result is the type specifier T.
The result of this function may not include all the apparent TYPE
declarations for VARIABLE. In particular, an implementation is free
to ignore TYPE declarations, only returning TYPE information which
was added to ENV by a call to AUGMENT-ENVIRONMENT.
Example:
This example assumes that the implementation records type
information in the environment, rather than ignoring it.
(DEFMACRO VARTYPE (VAR &ENVIRONMENT ENV)
`',(VARIABLE-TYPE VAR ENV))
(DEFVAR A 1)
(PROCLAIM '(FIXNUM A))
(DEFUN TEST ()
(LET ((B (AREF "X" 0))
(C 3))
(DECLARE (STRING-CHAR B))
(LIST (VARTYPE A) (VARTYPE B) (VARTYPE C))))
(TEST) -> (FIXNUM STRING-CHAR NIL)
FUNCTION-FTYPE function &optional env [Function]
FUNCTION is a function name. This function returns the functional
type specifier associated with the function in the environment, or
the symbol FUNCTION if there is no functional type declaration or
proclamation associated with the function.
The result of this function may not include all the apparent FTYPE
declarations for FUNCTION. In particular, an implementation is free
to ignore FTYPE declarations, only returning FTYPE information which
was added to ENV by a call to AUGMENT-ENVIRONMENT.
Example:
This example assumes that the implementation records ftype
information in the environment, rather than ignoring it.
(DEFMACRO FUNTYPE (FUN &ENVIRONMENT ENV)
`',(FUNCTION-FTYPE FUN ENV))
(DEFUN A-FUNCTION (X)
(+ X 3))
(PROCLAIM '(FTYPE (FUNCTION (FIXNUM) FIXNUM) A-FUNCTION))
(DEFUN TEST ()
(FLET ((ANOTHER-FUNCTION (X)
(+ X 2)))
(DECLARE (FTYPE (FUNCTION (INTEGER) INTEGER) ANOTHER-FUNCTION))
(LIST (FUNTYPE A-FUNCTION) (FUNTYPE ANOTHER-FUNCTION))))
(TEST) -> ((FUNCTION (FIXNUM) FIXNUM) (FUNCTION (INTEGER) INTEGER))
AUGMENT-ENVIRONMENT env &KEY variable
symbol-macro
function
macro
declare [Function]
This function returns a copy of ENV, augmented with the information
provided by the keyword arguments. The arguments are supplied as
follows:
:VARIABLE A list of symbols which shall be visible as bound
variables in the new environment. Whether each
binding is to be interpreted as special or lexical
depends on SPECIAL declarations recorded in the
environment or provided in the :DECLARE argument list.
:SYMBOL-MACRO A list of symbol macro definitions, in the same format
as the CADR of a SYMBOL-MACROLET special form. The
new environment will have local symbol-macro bindings
of each symbol to the corresponding expansion, so that
MACROEXPAND will be able to expand them properly.
:FUNCTION A list of function names which shall be visible as local
function bindings in the new environment.
:MACRO A list of local macro definitions, in the same format
as the CADR of a MACROLET special form. The new
environment will have local macro bindings of each
name to the corresponding expander function, which
will be returned by MACRO-FUNCTION and used by
MACROEXPAND.
:DECLARE A list of decl-specs. The new environment will
contain information about SPECIAL, TYPE, and FTYPE
declarations appearing within the list.
An error is signalled if any of the symbols naming macros in the
:SYMBOL-MACRO alist are also included in the :VARIABLE list.
An error is signalled if any of the names specified as keys in the
:MACRO alist are also included in the :FUNCTION list. The consequences
of destructively modifying the list structure of any of the arguments
to this function are undefined.
The extent of the returned environment is the same as the extent of
the argument.
While an environment argument from EVALHOOK may be used as the
environment argument for this function, the reverse is not true. If
an attempt is made to use the result of AUGMENT-ENVIRONMENT as
the environment argument for EVALHOOK, the consequences are undefined.
The environment returned by AUGMENT-ENVIRONMENT may only be used for
syntactic analysis, ie. the functions specified by this proposal and
functions such as MACROEXPAND.
[If PROCLAIM-LEXICAL is adopted, LEXICAL declarations would also
be recognized.]
Rationale:
This proposal defines a minimal set of accessors and a constructor
for environments.
The symbol-macro and macrolet definitions and declarations passed
to AUGMENT-ENVIRONMENT are in the same form as those which would
normally be encountered by a code-walker. This makes it easier to
use. In particular, there is no need for users to write their own
code to destructure macro arguments.
Making TYPE and FTYPE information optional continues to allow
implementations the freedom to simply ignore all such declarations.
Proposal (SYNTACTIC-ENVIRONMENT-ACCESS:MEDIUM):
This is the same as proposal SMALL, but also includes:
There are two kinds of environments, local and remote. Local
environments are created by EVAL and COMPILE, and are used in
situations where the information in the environment pertains
to the immediate running Lisp environment. Remote environments
are used by processors such as COMPILE-FILE to model what the
Lisp environment will look like when the code being processed
is actually loaded.
If AUGMENT-ENVIRONMENT receives a remote environment as an argument,
then the new environment returned by this function will also be
remote, and the two will refer to the same model of the remote
environment.
ENVIRONMENT-REMOTE-P env [Function]
Returns true if ENV is a remote environment, false otherwise.
WITH-REMOTE-ENVIRONMENT var &body body [Macro]
Evaluates the BODY forms with VAR bound to a newly created remote
environment. The extent of the environment is the dynamic extent of
the WITH-REMOTE-ENVIRONMENT form.
This is the only specified mechanism by which a new remote
environment may be created.
Rationale:
The notion of local and remote environments may be useful for
developing the CLOS meta-object protocol. At some future time,
we may wish to tighten the specification of how compile-time
definitions of defining macros such as DEFMACRO or DEFCLASS are
achieved, by saying that the compile-time definitions must be made
only in the remote environment.
Proposal (SYNTACTIC-ENVIRONMENT-ACCESS:LARGE):
This is the same as proposal MEDIUM, but also includes:
ENVIRONMENT-PROPERTY env name property &optional default
This function and its SETF method allow the association of arbitrary
'global' properties with names within an environment. An
environment can be thought of as having a local property list
associated with any name, and this function provides access to that
property list.
A remote environment may be thought of as an extension of the local
environment. Thus, when this function is applied to a remote
environment and the property is not found, the global local environment
is then searched.
The association between names and property lists uses EQUAL to match
names. The search of the property list uses EQ to match properties.
If the property is not found, DEFAULT is returned.
Using SETF of ENVIRONMENT-PROPERTY affects all environments which
refer to the same environment model. In particular, if ENV is a
local environment then all local environments are affected, while if
ENV is a remote environment, then all environments refering to the
same remote environment model as the argument are affected.
[Note: The local property list of a name is not necessarily the
symbol-plist of the name, though that is a possible implementation
for names which are symbols.
Note: The use of EQUAL as the matching function for names is to
allow for proposed extensions to function names. If no such
extension occurs, then EQ could be used instead.]
Rationale:
This would provide a mechanism for making and accessing global
definitions in the remote environment.
Cost to Implementors:
Most implementations already record some of this information in some
form. Providing these functions should not be too difficult, but it
is a more than trivial amount of work.
Cost to Users:
This change is upward compatible with user code.
Current practice:
No implementation provides all of this interface currently. Portable
Common Loops defines a subset of this functionality for its code
walker and implements it on a number of diffent versions of Common
Lisp. IIM uses the functionality provided by ENVIRONMENT-REMOTE-P
and ENVIRONMENT-PROPERTY (under other names) to implement the
association between names and remote metaobjects (macro and type
definitions, CLOS remote metaobjects, &etc).
Discussion:
The first version of this proposal expressly did not deal with the
objects which are used as environments by EVALHOOK. This version is
extended to support them in the belief that such environments share a
lot of functionality with the syntactic environments needed by a
compiler. While the two types of environments may have very
different implementations, there are many operations which are
reasonable to perform on either type, including all of the accessor
functions described by this proposal.
AUGMENT-ENVIRONMENT currently requires signaling an error when
symbol-macro names match variable names in the same call. This could
be reduced to "should signal". By requiring the error signaling, this
proposal is compatable with Proposal SYMBOL-MACROLET-DECLARE:ALLOW,
which says
"... signals an error if a SPECIAL declaration names one of the symbols
being defined as a symbol-macrolet."
Maintaining compatability with the SYMBOL-MACROLET-DECLARE proposal
allows fairly trivial implementations of the SYMBOL-MACROLET
special-form in terms of the AUGMENT-ENVIRONMENT function.
An possible alternative syntax for WITH-REMOTE-ENVIRONMENT might be
WITH-REMOTE-ENVIRONMENT (var &key) &body body
Can anyone suggest candidates for keyword options? We could do this
even if we can't think of any immediately, leaving room for
implementation-specific extensions. One candidate option that some
implementations might want would be to specify a target machine for
the compilation.
Kim Barrett originally suggested that WITH-COMPILATION-UNIT should
provide the mechanism for creating new remote environments. However,
it has been suggested that WITH-COMPILATION-UNIT is intended to serve
a somewhat different purpose.
Sandra Loosemore says:
I support proposal SMALL but would vote against both of the larger
proposals. It's true that they provide functionality which *might* be
useful to implement CLOS, but there is nothing now in the standard
that *requires* this functionality to be added. In fact, the version
of issue COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS that was accepted at
the January meeting explicitly leaves unspecified the mechanism by
which defining macros make definitions available to the compiler. We
have very little practical experience with using environment objects
for this purpose and I think it would be premature to try to
standardize it at this point. In particular, since the meta-object
protocol is still undergoing what appear to be substantial changes,
let's wait until it settles down and then see what kind of compiler
hooks it needs, instead of possibly standardizing the wrong thing.
-------
∂12-Mar-89 1022 CL-Compiler-mailer Issue CONSTANT-COMPILABLE-TYPES, version 7
Received: from lucid.com by SAIL.Stanford.EDU with TCP; 12 Mar 89 10:22:27 PST
Received: from challenger ([192.9.200.17]) by heavens-gate.lucid.com id AA01313g; Sun, 12 Mar 89 10:15:26 PST
Received: by challenger id AA06612g; Sun, 12 Mar 89 10:10:49 PST
Date: Sun, 12 Mar 89 10:10:49 PST
From: Richard P. Gabriel <rpg@lucid.com>
Message-Id: <8903121810.AA06612@challenger>
To: cperdue@Sun.COM
Cc: cl-compiler@sail.stanford.edu
In-Reply-To: Cris Perdue's message of Sat, 11 Mar 89 17:18:44 PST <8903120118.AA18680@clam.sun.com>
Subject: Issue CONSTANT-COMPILABLE-TYPES, version 7
The subset of functions disallowed are all compiled functions and
all interpreted functions with non-null lexical environments. There is
no predicate for this.
My thought is that, as closely as possible, the fact is irrelevant
that a compiler or dumper intervened between the production of the
code by a programmer into a well-formed program and the execution of
the code by a computer.
Therefore, variables are mutable, and EQness required by the standard
for conforming programs should be maintained. If there is a problem, I
think it is better to change the semantics of Common Lisp rather than
making compiling and loading special cases.
∂12-Mar-89 1031 CL-Compiler-mailer Issue CONSTANT-COMPILABLE-TYPES, version 7
Received: from lucid.com by SAIL.Stanford.EDU with TCP; 12 Mar 89 10:31:25 PST
Received: from challenger ([192.9.200.17]) by heavens-gate.lucid.com id AA01319g; Sun, 12 Mar 89 10:24:17 PST
Received: by challenger id AA06634g; Sun, 12 Mar 89 10:19:40 PST
Date: Sun, 12 Mar 89 10:19:40 PST
From: Richard P. Gabriel <rpg@lucid.com>
Message-Id: <8903121819.AA06634@challenger>
To: sandra%defun@cs.utah.edu
Cc: cperdue@Sun.COM, cl-compiler@sail.stanford.edu
In-Reply-To: Sandra J Loosemore's message of Sat, 11 Mar 89 19:03:47 MST <8903120203.AA00900@defun.utah.edu>
Subject: Issue CONSTANT-COMPILABLE-TYPES, version 7
I think we can ignore non-Lisp functions. Here's why. First, we're
specifying Common Lisp, not C, so that only user-written functions in
Common Lisp have defined behavior under this proposal. Second, we're
only specifying what happens to well-formed programs that use the
built-in Common Lisp functions and etc (in the old days, the 775 magic
symbols). A proposal I have on the CL-cleanup table is that any
alteration of these magic functions has undefined consequences, except
where explicitly allowed. Therefore, the only important part of a
built-in CL function is its name, which is easy to dump.
That is, if you only have to dump its name, it doesn't matter what
language the body is in, and no well-defined programs alter those
definitions.
This is an example of what I mean by a conservative approach.
-rpg-
∂12-Mar-89 1032 Common-Lisp-Object-System-mailer Issue: LOAD-OBJECTS (Version 3)
Received: from lucid.com by SAIL.Stanford.EDU with TCP; 12 Mar 89 10:32:40 PST
Received: from challenger ([192.9.200.17]) by heavens-gate.lucid.com id AA01322g; Sun, 12 Mar 89 10:25:36 PST
Received: by challenger id AA06637g; Sun, 12 Mar 89 10:20:59 PST
Date: Sun, 12 Mar 89 10:20:59 PST
From: Richard P. Gabriel <rpg@lucid.com>
Message-Id: <8903121820.AA06637@challenger>
To: sandra%defun@cs.utah.edu
Cc: CL-Cleanup@sail.stanford.edu, CL-Compiler@sail.stanford.edu,
Common-Lisp-Object-System@sail.stanford.edu
In-Reply-To: Sandra J Loosemore's message of Sat, 11 Mar 89 18:42:56 MST <8903120142.AA00878@defun.utah.edu>
Subject: Issue: LOAD-OBJECTS (Version 3)
Sandra's idea of two generic functions, one producing components and the
other doing construction/further initialization has some merits aside
from the question of dumping functions. I think we should consider it.
-rpg-
∂12-Mar-89 1045 CL-Compiler-mailer Re: Issue CONSTANT-COMPILABLE-TYPES, version 7
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 12 Mar 89 10:45:41 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA26164; Sun, 12 Mar 89 11:43:13 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA01463; Sun, 12 Mar 89 11:43:11 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903121843.AA01463@defun.utah.edu>
Date: Sun, 12 Mar 89 11:43:10 MST
Subject: Re: Issue CONSTANT-COMPILABLE-TYPES, version 7
To: Richard P. Gabriel <rpg@lucid.com>
Cc: sandra%defun@cs.utah.edu, cperdue@Sun.COM, cl-compiler@sail.stanford.edu
In-Reply-To: Richard P. Gabriel <rpg@lucid.com>, Sun, 12 Mar 89 10:19:40 PST
Hmmm. I suppose it would work to dump the troublesome functions by
name, assuming of course that you can determine what the name is.
What do you do for an anonymous function?
To me, the real question is whether there are benefits from dumping
constant functions that would justify all the work involved for
implementors. I guess I would take more seriously your arguments
about them deserving better treatment at the hands of the compiler if
you were also proposing to require them to have a PRINTed
representation such that they can be READ in again. After all, such
non-Lispy things as pathnames and arrays get better treatment than
functions....
-Sandra
-------
∂12-Mar-89 1111 CL-Compiler-mailer Issue CONSTANT-COMPILABLE-TYPES, version 7
Received: from lucid.com by SAIL.Stanford.EDU with TCP; 12 Mar 89 11:11:37 PST
Received: from challenger ([192.9.200.17]) by heavens-gate.lucid.com id AA01350g; Sun, 12 Mar 89 11:04:27 PST
Received: by challenger id AA06691g; Sun, 12 Mar 89 10:59:49 PST
Date: Sun, 12 Mar 89 10:59:49 PST
From: Richard P. Gabriel <rpg@lucid.com>
Message-Id: <8903121859.AA06691@challenger>
To: sandra%defun@cs.utah.edu
Cc: sandra%defun@cs.utah.edu, cperdue@Sun.COM, cl-compiler@sail.stanford.edu
In-Reply-To: Sandra J Loosemore's message of Sun, 12 Mar 89 11:43:10 MST <8903121843.AA01463@defun.utah.edu>
Subject: Issue CONSTANT-COMPILABLE-TYPES, version 7
Several (most?) implementations have back pointers from the compiled
function object to the symbol for built-in functions to aid debugging.
Nevertheless, many (all?) implementations encode the starting address
of the code object, which can easily (if it doesn't already) contain
all the information needed to retrieve the function name.
Printing functions? Well, the straightforward representation of them
already works more or less (list structure). Should we extend this?
Maybe, I haven't thought about it too much.
-rpg-
∂12-Mar-89 2014 Common-Lisp-Object-System-mailer Re: remote environments
Received: from Xerox.COM by SAIL.Stanford.EDU with TCP; 12 Mar 89 20:13:58 PST
Received: from Semillon.ms by ArpaGateway.ms ; 12 MAR 89 20:00:24 PST
Date: Sun, 12 Mar 89 20:00 PST
From: Gregor.pa@Xerox.COM
Subject: Re: remote environments
To: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>, Sandra J Loosemore
<sandra%defun@cs.utah.edu>, David N Gray <Gray@DSG.csc.ti.com>, Patrick
Dussud <dussud@lucid.com>
cc: Kim A. Barrett <IIM%ECLA@ECLC.USC.EDU>, cl-compiler@sail.stanford.edu,
common-lisp-object-system@sail.stanford.edu
Fcc: BD:>Gregor>mail>outgoing-mail-5.text.newest
In-Reply-To: <19890307023706.3.MOON@EUPHRATES.SCRC.Symbolics.COM>,
<19890309185905.3.MOON@EUPHRATES.SCRC.Symbolics.COM>,
<8903100017.AA10108@defun.utah.edu>,
<2814540528-3314903@Kelvin>,
<19890310174955.3.MOON@EUPHRATES.SCRC.Symbolics.COM>,
<8903101804.AA10718@defun.utah.edu>,
<8903101807.AA03577@challenger>,
<19890310194038.9.MOON@EUPHRATES.SCRC.Symbolics.COM>,
<8903101949.AA10788@defun.utah.edu>,
<8903102056.AA10809@defun.utah.edu>,
<8903102240.AA03941@challenger>,
<2814562184-4740019@Kelvin>,
<8903111615.AA00548@defun.utah.edu>,
<2814639239-9369595@Kelvin>,
<19890311223604.9.MOON@EUPHRATES.SCRC.Symbolics.COM>,
<19890311225826.2.MOON@EUPHRATES.SCRC.Symbolics.COM>,
<19890311230120.3.MOON@EUPHRATES.SCRC.Symbolics.COM>,
<19890311230257.4.MOON@EUPHRATES.SCRC.Symbolics.COM>,
<19890311232009.6.MOON@EUPHRATES.SCRC.Symbolics.COM>,
<19890311232035.7.MOON@EUPHRATES.SCRC.Symbolics.COM>,
<8903120019.AA00810@defun.utah.edu>
Message-ID: <19890313040005.0.GREGOR@SPIFF.parc.xerox.com>
Line-fold: no
In this message I question, once again, the existence of remote
metaobjects. I may be behind the times here, but since no one answered
the long message I sent about why we could do without remote metaobjects
I am still stuck at that point.
Date: Thu, 23 Feb 89 18:38:01 CST
From: David N Gray <Gray@DSG.csc.ti.com>
* Can a MAKE-INSTANCE be done by a macro expander, DEFCONSTANT, or "#."?
Date: Mon, 6 Mar 89 21:37 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
I think it would be much nicer if we could make compile-time classes
instantiable. However, I agree that it would not ruin the language to
omit that feature if we can't figure out how to do it.
Date: Fri, 10 Mar 89 16:49:44 CST
From: David N Gray <Gray@DSG.csc.ti.com>
> One way to look at this would be to
> say that it is implementation-dependent whether FINALIZE-INHERITANCE
> works or signals an error when given a class defined in the
> compile-time environment.
I don't understand these comments. It seems to me that the only real
questions are "is there a remote class object?" and "if so, how are
remote functions represented?".
If there is a remote class object then we must solve what I previously
previously called the `splicing' problem. That remote class object must
have pointers to other class objects which represent its superclasses.
If that is the case, then to some extent that remote class will be
instantiable. Any class that has direct supers can be finalized
provided that none of the supers are forward referenced. Since it can
be finalized, allocate-instance can be called. So, the only real
question is whether the initfunctions can be called. From the user's
point of view, that translates into whether make-instance can be called.
Date: Mon, 6 Mar 89 21:37 EST
From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
When you compile-file a DEFMETHOD, a method metaobject is created but it
is not added to the generic-function metaobject in the local environment.
Instead it is added to a different generic-function metaobject created
in the remote environment. That's my model of what has to happen. Note
that this should be completely consistent with the way that compile-file
of a DEFCLASS, with a direct superclass whose name is defined in the
local environment and not in the remote environment, does not add the
new class metaobject to the direct subclasses of the local superclass,
but rather to a different object. (I realize we haven't agreed on what
this paragraph says, or even seen a coherent proposal, yet. I'm just
telling you my model.)
Yes, this is the `splicing' problem which must be solved to have remote
metaobjects.
In the past there has been some controversy about whether the remote
environment can inherit from the local environment. I think this is
crystal clear: since some user-defined classes have STANDARD-OBJECT
as a direct superclass, and STANDARD-OBJECT is not defined in the same
file, the remote environment is clearly inheriting from the local
environment. Different implementations might want to address the
details of this differently, but I think it's clear that there has to
be provision for it in the metaobject model. It makes things more
complicated, but that's unavoidable.
More splicing problems.
Date: Fri, 10 Mar 89 16:49:44 CST
From: David N Gray <Gray@DSG.csc.ti.com>
> This is an interesting idea, but I think it's too restrictive. Here's a
> plausible and many-times proposed application for metaobjects which
> would not be possible if we adopted this idea. Suppose you made an
> optimizing compiler that is allowed to assume that no class
> redefinitions, no method redefinitions, and no newly-defined subclasses
> will be created at run time. The compiler is to take advantage of this
I certainly don't want to prevent any implementation from doing that.
The real issue is what is the minimal functionality that all
implementations must support.
As Moon says in a later message, the issue here is not what the
implementation can do, its what portable programs can do. It needs to
be the case that one can write a portable, metaobject `compiler'. Many
people have wanted to do this. People currently do this with PCL which
surely has no remote metaobjects.
But, do we need remote metaobjects to do this? I have argued before
that we don't. I believe it would be a reasonable restriction on
such portable programs that the program they are `compiling' be loaded
before it can be compiled. No one has shown me why that isn't the case.
But, what I know wonder is whether it would be a reasonable restriction
on implementors. Because, with this metaobject stuff, we can specify
minimal behavior, but we can't specify minimally. If we specify that
there are no remote metaobjects, then implementations are going to have
to prevent themselves from ever instantiating remote versions of
portable metaobjects. Moreover they will have to be certain that user's
can't get their hands on remote metaobjects. This will make life more
complicated for implementations, I don't know how much more complicated.
On the other hand, solving the splicing problem will also be complicated
for implementations, and for the specification as well.
-------
∂13-Mar-89 0653 Common-Lisp-Object-System-mailer Re: remote environments
Received: from cs.utah.edu by SAIL.Stanford.EDU with TCP; 13 Mar 89 06:53:16 PST
Received: from defun.utah.edu by cs.utah.edu (5.61/utah-2.1-cs)
id AA15174; Mon, 13 Mar 89 07:44:33 -0700
Received: by defun.utah.edu (5.61/utah-2.0-leaf)
id AA02017; Mon, 13 Mar 89 07:44:17 -0700
From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Message-Id: <8903131444.AA02017@defun.utah.edu>
Date: Mon, 13 Mar 89 07:44:15 MST
Subject: Re: remote environments
To: Gregor.pa@Xerox.COM
Cc: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>,
Sandra.J.Loosemore%defun@cs.utah.edu, <sandra%defun@cs.utah.edu>,
David N Gray <Gray@DSG.csc.ti.com>, Patrick%defun@cs.utah.edu,
Dussud <dussud@lucid.com>, Kim A. Barrett <IIM%ECLA@ECLC.USC.EDU>,
cl-compiler@sail.stanford.edu,
common-lisp-object-system@sail.stanford.edu
In-Reply-To: Gregor.pa@Xerox.COM, Sun, 12 Mar 89 20:00 PST
> Date: Sun, 12 Mar 89 20:00 PST
> From: Gregor.pa@Xerox.COM
>
> If there is a remote class object then we must solve what I previously
> previously called the `splicing' problem. That remote class object must
> have pointers to other class objects which represent its superclasses.
> If that is the case, then to some extent that remote class will be
> instantiable. Any class that has direct supers can be finalized
> provided that none of the supers are forward referenced. Since it can
> be finalized, allocate-instance can be called. So, the only real
> question is whether the initfunctions can be called. From the user's
> point of view, that translates into whether make-instance can be called.
At least three of us (Moon, Gray, and myself) appear to agree that
DEFMETHOD should not make the method function callable at compile time
(by analogy to DEFUN). If you're using the term "initfunctions" to
refer to things like the SHARED-INITIALIZE method, then no, they can't
be called at compile-time by default.
Of course, if you really those methods to be callable at compile time,
you can always wrap an explicit (EVAL-WHEN (EVAL COMPILE LOAD) ...)
around both the class and method definitions. Alternatively, you'd
probably put the definitions in another file and load it before
compiling the file that wants to instantiate that class at compile
time.
Personally, I think we could leave the issue of whether remote classes
exist or are instantiable unspecified for now. It would give those of
you who are trying to sort out the meta-object protocol more freedom,
and I don't think it would place an unreasonable burden on programmers.
> But, do we need remote metaobjects to do this? I have argued before
> that we don't. I believe it would be a reasonable restriction on
> such portable programs that the program they are `compiling' be loaded
> before it can be compiled. No one has shown me why that isn't the case.
I guess this is pretty much the same thing as I was saying above. The
one reservation I have (that others have also expressed) is that the
common practice of a file containing a class definition followed by
method definitions on that class should compile properly without
having to do any magic with pre-loading the file or wrapping
EVAL-WHENs around everything. Would that cause problems for the
meta-object protocol?
-Sandra
-------
∂13-Mar-89 0853 X3J13-mailer issue COMPILER-VERBOSITY, version 6
Received: from Think.COM by SAIL.Stanford.EDU with TCP; 13 Mar 89 08:53:16 PST
Return-Path: <barmar@Think.COM>
Received: from OCCAM.THINK.COM by Think.COM; Mon, 13 Mar 89 11:48:42 EST
Date: Mon, 13 Mar 89 11:50 EST
From: Barry Margolin <barmar@Think.COM>
Subject: issue COMPILER-VERBOSITY, version 6
To: cl-compiler@sail.stanford.edu
Cc: x3j13@sail.stanford.edu
In-Reply-To: <8903131546.AA02078@defun.utah.edu>
Message-Id: <19890313165001.9.BARMAR@OCCAM.THINK.COM>
I see the benefit of :PRINT, but do we really need :VERBOSE? What's the
difference between
(compile path :verbose t)
and
(format t "~&Compiling file ~A...~%" path)
(compile-file path)
(format t "~&done.~%")
If any users want to have this controlled by a global variable, they can
do what we (Thinking Machines) did years ago and package it up in their
own function. At this stage of the game I don't see the need to add
gratuitous features like this.
:PRINT is less gratuitous because it implements a feature that the user
can't add himself.
barmar
∂13-Mar-89 1023 CL-Compiler-mailer Re: issue MACRO-ENVIRONMENT-EXTENT, version 2
Received: from ti.com by SAIL.Stanford.EDU with TCP; 13 Mar 89 10:23:31 PST
Received: by ti.com id AA16774; Mon, 13 Mar 89 12:21:56 CST
Received: from Kelvin by tilde id AA26537; Mon, 13 Mar 89 12:14:31 CST
Message-Id: <2814804793-2851395@Kelvin>
Sender: GRAY@Kelvin.csc.ti.com
Date: Mon, 13 Mar 89 12:13:13 CST
From: David N Gray <Gray@DSG.csc.ti.com>
To: sandra%defun@cs.utah.edu (Sandra J Loosemore)
Cc: David N Gray <Gray@DSG.csc.ti.com>, cl-compiler@sail.stanford.edu,
"David A. Moon" <Moon@STONY-BROOK.SCRC.Symbolics.COM>,
"Kim A. Barrett" <IIM%ECLA@ECLC.USC.EDU>,
KMP@STONY-BROOK.SCRC.Symbolics.COM
Subject: Re: issue MACRO-ENVIRONMENT-EXTENT, version 2
In-Reply-To: Msg of Sat, 11 Mar 89 16:29:06 MST from sandra%defun@cs.utah.edu (Sandra J Loosemore)
> How about some wording like:
>
> The extent of the environment objects passed to MACROEXPAND or
> MACROEXPAND-1 by COMPILE-FILE, COMPILE, and EVAL is the dynamic extent
> during which the macro and its expansion are processed. In
> particular, the extent includes the expansion of any other macro calls
> appearing lexically within the form returned from MACROEXPAND or
> MACROEXPAND-1.
That would be fine.
> I think this would fit in with issue SYNTACTIC-ENVIRONMENT-ACCESS, but
> I'm still confused about why you would still need COPY-ENVIRONMENT.
COPY-ENVIRONMENT would only be needed if there are implementations using
stack list environments that disappear as soon as the macro expander
returns. Kent, does Symbolics need this?
> One other thing that's still not really clear to me is why you think
> this is better than just saying they have (or can be copied to have)
> indefinite extent. It would make the typed-variable example work but
> not be of much use for macro-caching. Can you provide a more specific
> rationale than just saying that it gives implementors more flexibility?
I want to be able to do something like this:
(defun VARIABLE-KIND (variable &optional environment)
(if (compiler:compilation-environment-p environment)
(case (compiler:var-type (compiler:lookup-var variable compiler:vars))
...)
;; else handle interpreter environment
...))
where the attributes of the variable are actually found in special
variable compiler:vars instead of in the environment object. This would
permit implementing proposal SYNTACTIC-ENVIRONMENT-ACCESS without
needing to redesign the compiler's data structures.